The Johns Hopkins University

Clear Guide Medical, Inc.

Strider Labs, Inc.

Amazon Technologies, Inc.

Leland Stanford Junior University

Gregory Donald Hager

Kelleher Guerin

Emad M Boctor

Pezhman Foroughi

Philipp Jakob Stolka

Matthew C Rendina

Eliot Leonard Wegbreit

Rajesh Kumar

Sebastian Riedel

Russell H Taylor

Peter Louis Gehlbach

James Tahara Handa

Sanjeev Khudanpur

Carol E Reiley

Henry C Lin

Nicolas Padoy

Jin Ung Kang

Gabor Fichtinger

Ejaz Ahmed

Ehsan Basafa

Seth D Billings

Balazs Peter Vagvolgyi

Raphael Sznitman

Zachary Pezzementi

Balakrishnan Varadarajann

Hassan Rivaz

Gopi Prashanth Gopal

Ioana Fleming

Allison Mariko Okamura

Marcin Arkadiusz Balicki

Andre David Brown

Abhay Doke

Venkataraman Santhanam

Balakrishnan Varadarajan

Harish Srinivasan

Masaru Ishii

Eric M Meisner

Daniel J Marota

Haluk N Tokgozoglu

Anand Viswanathan

Emad Mikhail Boctor

Emad Moussa Boctor

Foroughi Pezhman

Basafa Ehsan

Arnand Viswanathan

This disclosure describes, in part, techniques for identifying interactions between users and inventory locations. For instance, system(s) may receive image data generated by an imaging device located within a facility. The system(s) may then analyze the image data in order to determine one or more gaze points, where the gaze point(s) indicate location(s) on an inventory location for which a user was looking over a period of time. The system(s) may then generate gaze data indicating period(s) of time that the user was looking at the portion(s) of the inventory location. In some examples, the gaze data represents a heatmap. The system(s) may then use planogram data to determine identifier(s) of item(s) located at the portion(s) of the inventory location. Additionally, the system(s) may generate metrics data representing at least the identifier(s) of the item(s) and the period(s) of time that the user was looking at the item(s).

Identifying interactions with inventory locations

Provided is a method, device, and computer-readable medium for controlling a robot graphic user interface ('RGUI') on a mobile device. The method can include determining a distance, a position, or both of the mobile device with respect to a first robot; and causing, by a processor, a first RGUI to be displayed on a display of the mobile device based on the determining.

Method, device, and computer-readable medium for mobile device management of collaborative industrial robot

This disclosure describes, in part, techniques for identifying interactions and events associated with inventory locations. For instance, system(s) may receive image data representing a user interacting with an inventory location. The system(s) may then generate heatmap data indicating a first portion of the image data that represents the inventory location and feature data indicating a second portion of the image data that represents the user. Next, the system(s) may analyze the heatmap data with respect to the feature data to determine that the second portion of the image data corresponds to the first portion of the image data. As such, the system(s) may determine that the user is interacting with the inventory location. Based on the determination, the system(s) may analyze the first portion of the image data to identify an event that occurs at the inventory location, such as the user removing an item.

Detecting interactions with inventory locations

This disclosure describes, in part, a mobile apparatus for identifying items within a facility using an omnidirectional imaging system. For instance, the mobile apparatus may include a main frame, a chassis attached to the main frame, a basket that attaches to the chassis in order to prevent the basket from contacting the main frame, and an omnidirectional imaging system providing image data in and around the mobile apparatus. A user may place item(s) within a receptacle of the basket. The mobile apparatus may further include a handlebar module attached to the main frame, the handlebar module including the omnidirectional imaging system and a computing system for identifying items and events in and around the cart.

Omnidirectional item identifying mobile apparatus

Methods and systems for connection-driven generation of robotic user interfaces and modification of robotic properties include detecting a connection of a robotic peripheral to a robot; obtaining a peripheral property set corresponding to the robotic peripheral, wherein the peripheral property set includes one or more properties of the robotic peripheral; modifying, based on the peripheral property set, a robotic property set that includes one or more properties of the robot to provide a modified robotic property set; generating, during runtime, a robotic graphical user interface ('RGUI') dynamically based on the peripheral property set, wherein the RGUI provides at least one user-accessible interface to control the robot and the robotic peripheral; and controlling, based on the modified robotic property set, the robot and the robotic peripheral in response to user input received via the RGUI.

Generation of robotic user interface responsive to connection of peripherals to robot

System and methods to create an immersive virtual environment using a virtual reality system that receives parameters corresponding to a real-world robot. The real-world robot may be simulated to create a virtual robot based on the received parameters. The immersive virtual environment may be transmitted to a user. The user may supply input and interact with the virtual robot. Feedback such as the current state of the virtual robot or the real-world robot may be provided to the user. The user may train the virtual robot. The real-world robot may be programmed based on the virtual robot training.

Robot control, training and collaboration in an immersive virtual reality environment

A system and method for automatically computing desirable palm grasp configurations of an object by a robotic hand is disclosed. A description of the object's surface is obtained. A grasp configuration of a robotic hand comprising a palm and one or more fingers is selected, which specifies a hand configuration and joint variables of the hand, and a plurality of contact points on the object's surface for the palm and fingers. The coefficient of friction at each of the contact points is determined, and a description of one or more external wrenches to which may apply to the object is acquired. The ability of the robotic hand to hold the object against the external wrenches in the selected configuration is then computed. In some embodiments, a plurality of grasp configurations may be compared to determine which are the most desirable. In other embodiments, the space of hand configurations, or the smaller space of palm contact configurations, may be searched to find the most desirable grasp configurations.

System and method for computing grasps for a robotic hand with a palm

Methods and systems for enabling human-machine collaborations include a generalizable framework that supports dynamic adaptation and reuse of robotic capability representations and human-machine collaborative behaviors. Specifically, a method of feedback-enabled user-robot collaboration includes obtaining a robot capability that models a robot's functionality for performing task actions, specializing the robot capability with an information kernel that encapsulates task-related parameters associated with the task actions, and providing an instance of the specialized robot capability as a robot capability element that controls the robot's functionality based on the task-related parameters. The method also includes obtaining, based on the robot capability element's user interaction requirements, user interaction capability elements, via which the robot capability element receives user input and provides user feedback, controlling, based on the task-related parameters, the robot's functionality to perform the task actions in collaboration with the user input; and providing the user feedback including task-related information generated by the robot capability element in association with the task actions.

System and method for flexible human-machine collaboration

A system and method for constructing a 3D scene model comprising 3D objects and representing a scene, based upon a prior 3D scene model. The method comprises the steps of acquiring an image of the scene; initializing the computed 3D scene model to the prior 3D scene model; and modifying the computed 3D scene model to be consistent with the image. The step of modifying the computed 3D scene models consists of the sub-steps of comparing data of the image with objects of the 3D scene model, resulting in associated data and unassociated data; using the unassociated data to compute new objects that are not in the 3D scene model and adding the new objects to the 3D scene model; and using the associated data to detect objects in the prior 3D scene model that are absent and removing the absent objects from the 3D scene model. The present invention may also be used to construct multiple alternative 3D scene models.

System and method for constructing a 3D scene model from an image

Growing community of inventors

Baltimore, MD, United States of America

Gregory Donald Hager