Simbe Robotics, Inc.

Simbe Robotics, Inc.b

Jariullah Safi

Bradley Bogolea

Mirza Akbar Shah

Lorin Vandegrift

Jeffrey Gee

Durgesh Tiwari

Luke Fraser

Shiva Reddy

Kanchan Bahirat

Tom Gehani

Cem Ersoz

One variation of a method for maintaining perpetual inventory within a store includes: accessing a radar scan of an inventory structure within a store; accessing an optical image of the inventory structure; identifying a product type associated with the slot in a region of the optical image; retrieving a volumetric definition of the product type; locating a slot volume defining the slot in the radar scan; extracting a volumetric representation of a set of product units intersecting the slot volume in the radar scan; segmenting the volumetric representation by the volumetric definition to calculate a quantity of the set of product units occupying the slot; and updating a stock record of the store to reflect the quantity of the set of product units occupying the slot.

Method for maintaining perpetual inventory within a store

One variation of a method for segmenting scenes of product units arranged in inventory structures within a store includes: accessing an image based on data captured by a mobile robotic system; detecting a shelving segment in the image; reading a segment identifier from a segment tag, detected in the image, arranged on the shelving segment; accessing a product template representing a product type in the set of product types assigned to the shelving segment based on the segment identifier; detecting a set of product features, in the first region of the image. In response to detecting the set of product features analogous to features of the product template: confirming presence of the unit of the first product type on the shelf in the shelving segment and appending the first product type to a list of product types presently stocked in the shelving segment.

Method for scene segmentation

One variation of a method for deploying a mobile robotic system to scan inventory structures within a store includes: dispatching the mobile robotic system to navigate along inventory structures within the store during a setup cycle; at the mobile robotic system, while navigating along the inventory structures during the setup cycle, capturing a set of wireless connectivity metrics representing connectivity to a first wireless network; assembling the set of wireless connectivity metrics into a wireless connectivity map of the store; estimating a processing duration from start of the scan cycle to transformation of images of the inventory structures, captured by the mobile robotic system, into a stock condition of the store; and dispatching the mobile robotic system to autonomously capture images of the inventory structures within the store during a scan cycle preceding a scheduled restocking period in the store based on the processing duration.

Method for deploying a robotic system to scan inventory within a store based on local wireless connectivity

One variation of a method for tracking placement of products in a store includes: accessing an image recorded by a mobile robotic system within a store; detecting a shelf in a region of the image; based on an address of the shelf, retrieving a list of products assigned to the shelf by a planogram of the store; retrieving a set of template images—from a database of template images—defining visual features of products specified in the list of products; extracting a set of features from the region of the image; determining that a unit of the product is mis-stocked on the shelf in response to deviation between the set of features and features in a template image, in the set of template images, representing the product; and in response to determining that the unit of the product is mis-stocked on the shelf, generating a restocking prompt for the product.

Method for tracking placement of products on shelves in a store

One variation of a method for stock keeping in a store includes: accessing an image captured by a fixed camera within the store; retrieving a field of view of the fixed camera; estimating a segment of an inventory structure in the store depicted in the image based on a projection of the field of view onto a planogram of the store; identifying a set of slots within the inventory structure segment; retrieving a product model representing a set of visual characteristics of a product type assigned to a slot, in the set of slots, by the planogram; extracting a constellation of features from the image; if the constellation of features approximates the set of visual characteristics in the product model, detecting presence of a product unit of the product type occupying the inventory structure segment; and representing presence of the product unit, occupying the inventory structure segment, in a realogram.

Method for stock keeping in a store with fixed cameras

One variation of a method for tracking and maintaining inventory in a store includes: accessing a image of an inventory structure in the store; identifying a top shelf, in the inventory structure, depicted in the image; identifying a set of product units occupying the top shelf based on features detected in the image; identifying a second shelf, in the set of shelves in the inventory structure, depicted in the image, the second shelf arranged below the top shelf in the inventory structure; based on features detected in the image, detecting an understock condition at a slot—assigned to a product type—on the second shelf; and, in response to the set of product units comprising a product unit of the product type, generating a prompt to transfer the product unit of the product type from the top shelf into the slot on the second shelf at the inventory structure.

Method for tracking and maintaining inventory in a store

One variation of a method for tracking promotional states of slots in inventory structures within a store includes: accessing an image of an inventory structure within a store; detecting a shelf tag on the inventory structure in the image; extracting a set of features from the shelf tag detected in the image; detecting a promotional tag on the inventory structure in the image; extracting a set of features from the promotional tag detected in the image; detecting a deviation between the shelf tag and the promotional tag based on a difference between the sets of features; and, in response to detecting the deviation between the shelf tag and the promotional tag, identifying the first promotional tag as erroneous, and notifying a store associate to replace the first promotional tag with a second promotional tag at the first slot, the second promotional tag correcting the difference.

Method for tracking and maintaining promotional states of slots in inventory structures within a store

A method for stock keeping in a store includes: accessing an image captured by a fixed camera within the store; retrieving a field of view of the fixed camera; estimating a segment of an inventory structure in the store depicted in the image based on a projection of the field of view onto a planogram of the store; identifying a set of slots within the inventory structure segment; retrieving a product model representing a set of visual characteristics of a product type assigned to a slot, in the set of slots, by the planogram; extracting a constellation of features from the image; if the constellation of features approximates the set of visual characteristics in the product model, detecting presence of a product unit of the product type occupying the inventory structure segment; and representing presence of the product unit, occupying the inventory structure segment, in a realogram.

A method for maintaining inventory within a store includes: accessing an image (e.g., a color image, depth image) depicting an inventory structure; detecting a slot region of the image depicting a slot; identifying a product type assigned to the slot; accessing a product dimension of the product type; defining a target region within the slot in the image based on the product dimension; defining a product region within the slot in the image based on the product dimension and the target region; defining a back-of-shelf plane intersecting the target region of the image; detecting a surface within the product region; and, in response to the surface intersecting the back-of-shelf plane, identifying the slot as empty and generating a prompt to restock the slot with product units of the product type.

Method for maintaining inventory in a store

One variation of a method for tracking placement of products in a store includes: accessing an image depicting a shelving structure in the store; reading a product identifier of a product from a shelf tag detected in the image; accessing a set of template features associated with the product identifier; based on the shelf tag detected in the image, locating a region in the first image depicting a slot on the shelving structure; confirming presence of a unit of the product in the slot in response to detecting features analogous to the set of template features in the region in the image; detecting absence of the product from the slot based on absence of features analogous to the set of template features in the region in the image; and in response to detecting absence of the product from the slot, generating a prompt to restock the slot.

One variation of a method for managing virtual shopping lists includes: dispatching robotic systems, deployed in store, to autonomously scan inventory structures within this store; deriving current stock conditions of this store based on scan data recorded by these robotic systems; initializing a virtual shopping list for a user; in response to receipt of selection of a first product, from a population of products, isolating a subset of stores, in the set of stores in the geographic region, associated with current stock conditions indicating presence of the first product and products previously added to the virtual shopping list; in response to the subset of stores including at least one store, adding a first identifier of the first product to the virtual shopping list; and specifying a particular store, in the subset of stores, for fulfillment of a set of products on the virtual shopping list.

Method for managing click and delivery shopping events

One variation of a method for tracking fresh produce in a store includes: accessing a first hyper-spectral image, of a produce display in a store, recorded at a first time; extracting a first spectral profile from a first region of the first hyper-spectral image depicting a first set of produce units in the produce display; identifying a first varietal of the first set of produce units; characterizing qualities (e.g., ripeness, bruising, spoilage) of the first set of produce units in the produce display based on the first spectral profile; and, in response to qualities of the first set of produce units in the produce display deviating from a target quality range assigned to the first varietal, generating a prompt to audit the first set of produce units in the produce display.

Method for tracking and characterizing perishable goods in a store

One variation of a method for automatically generating waypoints for imaging shelves within a store includes: dispatching a robotic system to autonomously generating a map of a floor space within the store; accessing an architectural metaspace defining target locations and addresses of the set of shelving structures within the store; distorting the architectural metaspace into alignment with the map to generate a normalized metaspace representing real locations and addresses of the set of shelving structures in the store; defining a set of waypoints distributed longitudinally along and offset laterally from a first shelving structure represented in the normalized metaspace based on a known position of an optical sensor in the robotic system; and dispatching the robotic system to record optical data while occupying the set of waypoints during an imaging routine.

Growing community of inventors

San Francisco, CA, United States of America

Jariullah Safi