Target Brands, Inc.

Arne Wilkin

Todd A Hagen

Donnie Scott Tolbert

Andrew James Wipf

Nicholas Lojewski

John Ojanen

Yeshwanth Gowda

Yeshwanth M Gowda

Donnie Tolbert

Disclosed are systems and techniques for determining out of stock conditions on shelves. The techniques can include receiving, by a computing system, image data from a camera having pixel locations that each uniquely address and store a pixel value, generating a backing map having cell locations that each uniquely address and share a unique address with a corresponding pixel location in the image data, each cell location storing a backing value being an empty value if the pixel value is classified as showing the backing of a shelf and the backing value being a nonempty value if the pixel value is classified as not showing the backing of the shelf, determining, in the backing map, a shelf area representing a location of the captured shelf, and identifying an empty area by finding an area above the shelf area containing a threshold number of cell locations storing the empty value.

Mobile apparatus with computer vision elements for classifying shelf-space

Described herein are systems and techniques for imaging inventory objects in an environment. A system can include a cart, a first fixed camera fixedly mounted on the cart at a first angle, a pan-tilt-zoom (PTZ) camera controllably mounted on the cart, a PTZ controller, and a cart controller. The PTZ controller can receive PTZ instructions from the cart controller and send engagement instructions to the PTZ camera. The cart controller can receive, from the first fixed camera, first image data that captures a first inventory object, determine, from the first image data, a spatial location of a first inventory object, generate PTZ instructions to cause the PTZ camera to capture the first inventory object, transmit the PTZ instructions to the PTZ controller, and receive PTZ image data that captures the first inventory object.

Mobile apparatus with computer vision elements for inventory condition

The disclosed technology provides for reconstructing products that are added to a shopping cart in three-dimensional space. The reconstructed products can be used for product validation. A shopping cart can include product validation hardware including light emitters and light sensors along a top perimeter of the shopping cart. The shopping cart can activate the light emitters to emit light across a top horizontal plane of the shopping cart, receive, from light sensors on a side of the shopping cart opposite a side where the activated light emitter is located, light intensity data as a product passes through the top horizontal plane and obstructs at least a portion of the emitted light from being detected by the light sensors, identify, from the light intensity data, slices of the product, and reconstruct the product in 3D space based on stitching together the slices.

Shopping cart with infrared product validation

In some implementations, a system for verifying items in a retail environment includes a physical shopping cart including a first set of sensors, and an automated checkout station including a second, different set of sensors. The physical shopping cart receives item verification data for verifying an item, detects the item as it enters the physical shopping cart, and performs a primary verification of the item. The automated checkout station obtains a virtual shopping cart that corresponds to the physical shopping cart. The virtual shopping cart includes a list of items that have been placed in the physical shopping cart, and a verification status of each item. The second, different set of sensors generate station sensor data that represents the physical shopping cart and the items in the physical shopping cart. A secondary verification of the physical shopping cart and its contents is performed by the automated checkout station.

Verification of items by automated checkout system

In some implementations, a method performed by data processing apparatuses includes receiving, by an item verification engine, a scan notification indicating that a mobile computing device has scanned an item. In response to receiving the scan notification, the item verification engine provides an instruction for a cart computing system to activate item sensors on a shopping cart. The item verification engine receives sensor data collected by the item sensors as a result of the item having been placed in the shopping cart. The item verification engine receives item verification data for verifying the scanned item from the mobile computing device, performs a verification of the item based on the sensor data and the item verification data, and provides verification results for presentation by the mobile computing device.

Verification of items placed in physical shopping cart

Disclosed are systems and techniques for identifying product identifiers in images. A technique can include receiving, by an edge computing device, x and y coordinates for a location of an out of stock shelf section, determining a frame of reference (FOR) based on the coordinates, the FOR including a location of a product identifier for the out of stock shelf section, determining incremental adjustments to a camera based on the FOR, instructing the camera to adjust by the incremental adjustments and capture an image of the location of the product identifier, performing image analysis on the image to identify the product identifier, determining second incremental adjustments to the camera to focus the camera on the location of the product identifier if the product identifier is not identifiable in the image, and instructing the camera to reset to original camera settings if the product identifier is identifiable in the image.

Growing community of inventors

Minneapolis, MN, United States of America

Arne Wilkin