Accel Robotics Corporation

John Quinn

Marius Buibas

Csaba Petre

Kaylee Feigum

Michael Brandon Maseda

Martin Alan Cseh

Chin-Chang Kuo

Aleksander Bapst

Tanuj Pankaj

Filip Lukasz Piekniewski

Jose Miguel Rodriguez

Mark Graham

Christopher Lai

Soheyl Yousefisahi

System that facilitates rapid onboarding of an autonomous (cashier-less) store by capturing images of the store's items from multiple angles, with varying background colors, and that builds a classifier training dataset from these images. Background surfaces may for example be coated with retroreflective tape or film, and variable-color incident light sources may generate the desired background colors. Embodiments may automatically rotate or otherwise reorient the item placed in the onboarding system, so that a relatively small number of cameras can capture views from multiple angles. When an item is placed in the system, a fully automated process may generate a sequence of item orientations and background colors, and may capture and process images from the cameras to create training images. Images of the item from multiple angles, under varying lighting conditions, may be captured without requiring an operator to move or reorient the item.

Multi-lighting conditions rapid onboarding system for visual item classification

System that facilitates rapid onboarding of an autonomous (cashier-less) store by capturing images of the store's items from multiple angles, with varying backgrounds, and that builds a classifier training dataset from these images. The system may have cameras in different positions, and backgrounds that generate different background colors. It may have a platform for items that can be switched between a transparent and non-transparent state; cameras below the platform may therefore capture images of the bottom side of the item when the platform is transparent. When an item is placed in the imaging system, a fully automated process may generate a sequence of background colors and may capture and process images from all of the cameras to create training images. Images of the item from multiple angles, including views of the entire external surface of the item, may be captured without requiring an operator to move or reorient the item.

Multi-angle rapid onboarding system for visual item classification

A projected image item tracking system that analyzes projected camera images to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed on them may then be attributed to a shopper near the area. Projected images may be combined to generate a 3D volume difference between the state of the area before and after shopper interaction. The volume difference may be calculated using plane-sweep stereo, or using convolutional neural networks. Because these methods may be computationally intensive, the system may first localize a change volume where items appear to have been displaced, and then generate a volume difference only within that change volume. This optimization results in significant savings in power consumption and in more rapid identification of items. The 3D volume difference may also indicate the quantity of items displaced, for example from a vertical stack.

Multi-surface image projection item tracking system

Person and projected image item tracking system

A sensor bar shelf monitor, for example that may be added to an existing shelving system to convert a store to autonomous operation. The sensor bar may contain distance sensor to detect shoppers reaching towards items on a shelf, and cameras to determine which items shoppers have taken. It may be installed into shelf supports such as gondola shelving uprights. The sensor bar may be located at the front edge of a shelf, and may monitor the shelf below. Placing the sensor bar along the front edge prevents damage to electronics from spills or shelf cleaning, and prevents heat from the sensor bar electronics from damaging items on the shelf. The sensor bar may have a local sensor bar processor that collects sensor data; images may be analyzed locally or transferred to more powerful store processors. A sensor bar may also have controllable lights and controllable electronic labels.

Sensor bar shelf monitor

A system that integrates camera images and quantity sensors to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed may then be attributed to a shopper near the area. Shelves may be divided into storage zones, such as bins or lanes, and a quantity sensor may measure the item quantity in each zone. Quantity changes indicate that a shopper has taken or placed items in the zone. Distance sensors, such as LIDAR, may be used for shelves that push items towards the front. Strain gauges may be used for bins or hanging rods. Quantity changes may trigger analysis of camera images of the shelf to identify the items taken or replaced. Images from multiple cameras that view a shelf may be projected to a vertical plane at the front of the shelf to simplify analysis.

Camera-based tracking and authorization extension system

System that facilitates rapid onboarding of an autonomous (cashier-less) store by capturing images of items in the store's catalog from different angles, with varying backgrounds and lighting conditions, and that automatically builds a classifier training dataset from these images. The system may have cameras in different positions, lights supporting variable illumination, and monitor screens that generate different background colors. It may have an input device such as a barcode reader, and an operator terminal that prompts operators to place items into the imaging system in the necessary orientations. Once an item is placed in the imaging system, a fully automated process may generate a sequence of background colors, a sequence of lighting conditions, and may capture and process images from all of the cameras to create training images. Training images for an item may be generated in seconds, compared to many minutes per item using manual image capture and processing.

Rapid onboarding system for visual item classification

Smart shelf system that integrates images and quantity sensors

Projected image item tracking system

An autonomous store system that analyzes camera images to track people and their interactions with items using a processor that obtains a 3D model of a store that contains items and item storage areas. Receives images from cameras captured over a time period and analyzes the images and the 3D model of the store to detect a person in the store based on the images, calculates a trajectory of the person, identifies an item storage area proximal to the trajectory of the person during an interaction time period, analyzes two or more images to identify an item within the item storage area that is moved during the interaction time period. The images are captured within or proximal in time to the interaction time period, and the images contain views of the item storage area, and attribute motion of the item to the person. Enables calibration and placement algorithms for cameras.

Autonomous store system that analyzes camera images to track people and their interactions with items

A system that analyzes camera images to track a person in an autonomous store, and to determine when a tracked person takes or moves items in the store. The system may associate a field of influence volume around a person's location; intersection of this volume with an item storage area, such as a shelf, may trigger the system to look for changes in the items on the shelf. Items that are taken from, placed on, or moved on a shelf may be determined by a neural network that processes before and after images of the shelf. Person tracking may be performed by analyzing images from fisheye ceiling cameras projected onto a plane horizontal to the floor. Projected ceiling camera images may be analyzed using a neural network trained to recognize shopper locations. The autonomous store may include modular ceiling and shelving fixtures that contain cameras, lights, processors, and networking.

Autonomous store tracking system

A system that analyzes camera images to track a person from a point where the person obtains an authorization to a different point where the authorization is used. The authorization may be extended in time and space from the point where it was initially obtained. Scenarios enabled by embodiments include automatically opening a locked door or gate for an authorized person and automatically charging items taken by a person to that person's account. Supports automated stores that allow users to enter, take products and exit without explicitly paying. An illustrative application is an automated, unmanned gas station that allows a user to pay at the pump and then enter a locked on-site convenience store or a locked case with products the user can take for automatic purchase. Embodiments may also extend authorization to other people, such as occupants of the same vehicle.

Growing community of inventors

San Diego, CA, United States of America

John Quinn