Simbe Robotics, Inc.

Microsoft Technology Licensing, LLC

Jeffrey Gee

Bradley Bogolea

Mirza Akbar Shah

Jariullah Safi

Luke Fraser

Deepak Kumar

Lorin Vandegrift

Gungor Polatkan

Rohan Ramanath

Konstantin Salomatin

Mahesh Subhash Joshi

Durgesh Tiwari

Shivani Rao

Rupesh Gupta

Shipeng Yu

Onkar Anant Dalal

Fares Hedayati

Sara Smoot Gerrard

Lingjie Weng

Deepak Kumar Dileep Kumar

Jiaqi Ge

Yulia Astakhova

Vladislav Tcheprasov

Christopher Wright Lloyd, Ii

Scott Khamphoune

Vasudeva Nagaraja

Himanshu Khurana

Vijay Ramamurthy

One variation of a method for tracking placement of products in a store includes: accessing an image recorded by a mobile robotic system within a store; detecting a shelf in a region of the image; based on an address of the shelf, retrieving a list of products assigned to the shelf by a planogram of the store; retrieving a set of template images—from a database of template images—defining visual features of products specified in the list of products; extracting a set of features from the region of the image; determining that a unit of the product is mis-stocked on the shelf in response to deviation between the set of features and features in a template image, in the set of template images, representing the product; and in response to determining that the unit of the product is mis-stocked on the shelf, generating a restocking prompt for the product.

Method for tracking placement of products on shelves in a store

One variation of a method for tracking stock level within a store includes: dispatching a robotic system to image shelving structures within the store during a scan cycle; receiving images from the robotic system, each image recorded by the robotic system during the scan cycle and corresponding to one waypoint within the store; identifying, in the images, empty slots within the shelving structures; identifying a product assigned to each empty slot based on product location assignments defined in a planogram of the store; for a first product of a first product value and assigned to a first empty slot, generating a first prompt to restock the first empty slot with a unit of the first product during the scan cycle; and, upon completion of the scan cycle, generating a global restocking list specifying restocking of a set of empty slots associated with product values less than the first product value.

Method for tracking stock level within a store

In some embodiments, a computer system generates a recommendation for a user of an online service based on user actions that have been performed by the user within a threshold amount of time before the generation of the recommendation. For each user action, the computer system determines an intent classification that identifies an activity of the user and that corresponds to different types of user actions, as well as a preference classification that identifies a target of the activity, and then stores these intent and preference classifications as part of indications of the user actions for use in generating different types of recommendations using different types of recommendation models. Additionally, the computer system may use mini-batches of data from an incoming stream of logged data to train an incremental update to one or more recommendation models.

Recommendations using session relevance and incremental learning

One variation of a method for stock keeping in a store includes: accessing an image captured by a fixed camera within the store; retrieving a field of view of the fixed camera; estimating a segment of an inventory structure in the store depicted in the image based on a projection of the field of view onto a planogram of the store; identifying a set of slots within the inventory structure segment; retrieving a product model representing a set of visual characteristics of a product type assigned to a slot, in the set of slots, by the planogram; extracting a constellation of features from the image; if the constellation of features approximates the set of visual characteristics in the product model, detecting presence of a product unit of the product type occupying the inventory structure segment; and representing presence of the product unit, occupying the inventory structure segment, in a realogram.

Method for stock keeping in a store with fixed cameras

A method for stock keeping in a store includes: accessing an image captured by a fixed camera within the store; retrieving a field of view of the fixed camera; estimating a segment of an inventory structure in the store depicted in the image based on a projection of the field of view onto a planogram of the store; identifying a set of slots within the inventory structure segment; retrieving a product model representing a set of visual characteristics of a product type assigned to a slot, in the set of slots, by the planogram; extracting a constellation of features from the image; if the constellation of features approximates the set of visual characteristics in the product model, detecting presence of a product unit of the product type occupying the inventory structure segment; and representing presence of the product unit, occupying the inventory structure segment, in a realogram.

One variation of a method for tracking placement of products in a store includes: accessing an image depicting a shelving structure in the store; reading a product identifier of a product from a shelf tag detected in the image; accessing a set of template features associated with the product identifier; based on the shelf tag detected in the image, locating a region in the first image depicting a slot on the shelving structure; confirming presence of a unit of the product in the slot in response to detecting features analogous to the set of template features in the region in the image; detecting absence of the product from the slot based on absence of features analogous to the set of template features in the region in the image; and in response to detecting absence of the product from the slot, generating a prompt to restock the slot.

Techniques for implementing a feature generation pipeline for machine learning are provided. In one technique, multiple jobs are executed, each of which computes a different set of feature values for a different feature of multiple features associated with videos. A feature registry is stored that lists each of the multiple features. After the jobs are executed and the feature registry is stored, a model specification is received that indicates a set of features for a model. For each feature in a subset of the set of features, a location is identified in storage where a value for said each feature is found and the value for that feature is retrieved from the location. A feature vector is created that comprises, for each feature in the set of features, the value that corresponds to that feature. The feature vector is used to train the model or as input to the model.

Feature generation pipeline for machine learning

One variation of a method for automatically generating waypoints for imaging shelves within a store includes: dispatching a robotic system to autonomously generating a map of a floor space within the store; accessing an architectural metaspace defining target locations and addresses of the set of shelving structures within the store; distorting the architectural metaspace into alignment with the map to generate a normalized metaspace representing real locations and addresses of the set of shelving structures in the store; defining a set of waypoints distributed longitudinally along and offset laterally from a first shelving structure represented in the normalized metaspace based on a known position of an optical sensor in the robotic system; and dispatching the robotic system to record optical data while occupying the set of waypoints during an imaging routine.

Method for automatically generating waypoints for imaging shelves within a store

Techniques for improving the accuracy, relevancy, and efficiency of a computer system of an online service by providing a user interface to optimize a digital page of a user on the online service are disclosed herein. In some embodiments, a computer system receives a plurality of phrases for a type of job, selects a group of phrases from the plurality of phrases based on a corresponding relevancy measurement and a corresponding diversity measurement for each phrase in the selected group of phrases, and generates a recommendation for a page of a first user based on the selected group of phrases, with the recommendation comprising a suggested addition of the selected group of phrases to the page of the first user.

Phrase extraction for optimizing digital page

The video recommendation system provided with an on-line connection system generates on-line video recommendations using collaborative filtering for clusters of member profiles. The recommendation system clusters member profiles using member profile information as clustering criteria. The video recommendations are then generated for a given cluster, based on aggregation of video viewing history recorded for the member profiles that are in the given cluster, using the video similarity matrix. In order to produce video recommendations for a particular member profile, the recommendation system first determines cluster membership for the member profile, retrieves recommendations generated for that cluster, and provides recommendations to the associated member. A user interface including references to one or more recommended videos is rendered on a display device of a viewer.

Cluster-based collaborative filtering

Techniques for improving the accuracy, relevancy, and efficiency of a computer system of an online service by providing a user interface to optimize a digital page of a user on the online service are disclosed herein. In some embodiments, a computer system receives a plurality of phrases, and then, for each one of the plurality of phrases, selects a corresponding section of a page of a first user to suggest for placement of the phrase from amongst a plurality of sections using a placement classifier, and generates a corresponding recommendation for the page of a first user based on the phrase and the determined corresponding section of the page of the first user, with the recommendation comprising a suggested addition of the phrase to the determined corresponding section of the page of the first user.

Phrase placement for optimizing digital page

One variation of a method for tracking stock level within a store includes: at a robotic system, navigating along a first inventory structure in the store, broadcasting radio frequency interrogation signals according to a first set of wireless scan parameters, and recording a first set of wireless identification signals returned by radio frequency identification tags coupled to product units arranged on the first inventory structure; generating a first list of product units arranged on the first inventory structure based on the first set of wireless identification signals; detecting a first product quantity difference between the first list of product units and a first target stock list assigned to the first inventory structure by a planogram of the store; and generating a stock correction prompt for the first inventory structure in response to the first product quantity difference.

Techniques for improving the accuracy, relevancy, and efficiency of a computer system of an online service by providing a user interface to optimize a digital page of a user on the online service are disclosed herein. In some embodiments, a computer system identifies job postings published on an online service as corresponding to a type of job based on feature data of each one of the job postings, extracts phrases from the identified job postings based on a corresponding relevancy measurement and a corresponding diversity measurement for each one of the phrases, determines a corresponding section of a page of a user to suggest for placement of the extracted phrase using a placement classifier for each one of the extracted phrases, and generates a corresponding recommendation for the page based on the extracted phrase and the determined section of the extracted phrase for each one of the phrases.

User interface for optimizing digital page

One variation of a method for automatically generating a planogram for a store includes: dispatching a robotic system to autonomously navigate within the store during a mapping routine; accessing a floor map of the floor space generated by the robotic system from map data collected during the mapping routine; identifying a shelving structure within the map of the floor space; defining a first set of waypoints along an aisle facing the shelving structure; dispatching the robotic system to navigate to and to capture optical data at the set of waypoints during an imaging routine; receiving a set of images generated from optical data recorded by the robotic system during the imaging routine; identifying products and positions of products in the set of images; and generating a planogram of the shelving segment based on products and positions of products identified in the set of images.

Growing community of inventors

San Francisco, CA, United States of America

Jeffrey Gee