Google Inc.

Dmitry Kalenichenko

Sashikanth Chandrasekaran

Timothy Raymond Zwiebel

Varouj Armen Chitilian

Denise Ho

Michal Palczewski

Gerhard Florian Schroff

Menglong Zhu

Bo Chen

Andrew Gerald Howard

James William Philbin

Yinxiao Li

Marie Charisse White

Marco Andreetto

Mason Liu

Keren Ye

Weijun Wang

Tobias Christoph Weyand

Jumana Al Hashal

Systems and methods for detecting objects in a video are provided. A method can include inputting a video comprising a plurality of frames into an interleaved object detection model comprising a plurality of feature extractor networks and a shared memory layer. For each of one or more frames, the operations can include selecting one of the plurality of feature extractor networks to analyze the one or more frames, analyzing the one or more frames by the selected feature extractor network to determine one or more features of the one or more frames, determining an updated set of features based at least in part on the one or more features and one or more previously extracted features extracted from a previous frame stored in the shared memory layer, and detecting an object in the one or more frames based at least in part on the updated set of features.

Memory-guided video object detection

A merchant and a user register with a payment processing system, which establishes a facial template based on a user image. The user signs into a payment application via a user computing device, which receives an identifier from a merchant beacon device to transmit to the payment processing system. The payment processing system transmits facial templates to the merchant camera device for other users who are also signed in to the payment application in range of the merchant beacon device. The merchant camera device compares a captured facial image against the received facial templates to identify the user. A merchant POS device operator selects an account of the user. The merchant POS device transmits transaction details to the payment processing system, which processes the transaction with an issuer system. The payment processing system receives an approval of the transaction authorization request and transmits a receipt to the merchant POS device.

Identifying consumers in a transaction via facial recognition

An account management system establishes an account for a user. The user enters user account information into the account and the account management system establishes a facial template for the user based on an image of the face of the user. The user requests to change user account information at a merchant POS (POS) device. The merchant POS device captures a facial image of the user and transmits the image the account management system, which generates a facial template and compares the generated facial template against the existing facial template associated with user account. If the generated facial template is less than a threshold difference from the existing facial template, the user may update user account information at the merchant POS device, which communicates the updated user account information to the account management system. The account management system associates the updated user account information with the user account.

Facial profile password to modify user account data for hands free transactions

The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed 'MobileNets' for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.

Efficient convolutional neural networks and techniques to reduce associated computational costs

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an object embedding system. In one aspect, a method comprises providing selected images as input to the object embedding system and generating corresponding embeddings, wherein the object embedding system comprises a thumbnailing neural network and an embedding neural network. The method further comprises backpropagating gradients based on a loss function to reduce the distance between embeddings for same instances of objects, and to increase the distance between embeddings for different instances of objects.

Generating object embeddings from images

An account management system establishes a facial template for a user based on an image. The user computing device, signed into a payment application at the merchant location, receives an identifier from a merchant beacon device to transmit to the account management system, which transmits payment tokens based on payment account data and facial templates to the merchant POS device for each user signed in at the merchant location. The merchant POS device identifies the user by comparing a captured image of the user against the received facial templates and transmits the payment token to an issuer system. At a later time, the account management system receives, from a user computing device, a subsequent user image and generates a subsequent facial template. If the difference of the subsequent facial template is less than a threshold from the existing facial template, the subsequent facial template is associated with the user account.

Facial profile modification for hands free transactions

A user signs into an application via a user computing device at a merchant system location. The user computing device receives an identifier from a beacon device at the location to transmit to an account management system. The account management system transmits facial templates to the merchant camera device for users who are signed in to the application in range of the merchant beacon device. The user approaches a point of sale device to purchase a restricted product or service. The merchant camera device compares a captured facial image against the received facial templates to identify the user. A merchant POS device operator, after determining that the user is eligible for the restricted product or service based on account information of the user, selects a payment account of the user. The merchant POS device transmits transaction details to the account management system, which processes the transaction with an issuer system.

Identifying consumers via facial recognition to provide services

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating numeric embeddings of images. One of the methods includes obtaining training images; generating a plurality of triplets of training images; and training a neural network on each of the triplets to determine trained values of a plurality of parameters of the neural network, wherein training the neural network comprises, for each of the triplets: processing the anchor image in the triplet using the neural network to generate a numeric embedding of the anchor image; processing the positive image in the triplet using the neural network to generate a numeric embedding of the positive image; processing the negative image in the triplet using the neural network to generate a numeric embedding of the negative image; computing a triplet loss; and adjusting the current values of the parameters of the neural network using the triplet loss.

Growing community of inventors

Marina Del Rey, CA, United States of America

Dmitry Kalenichenko