Toyota Research Institute, Inc.

Toyota Motor Engineering & Manufacturing North America, Inc.

Other

Waymo LLC

Allison Thackston

Ron Goldman

Laura Stelzner

Samuel Zapolsky

Katarina Bouma

Sam Zapolsky

Katarina Miller

Matthew W Amacker

Arshan Poursohi

Christopher P Lee

Douglas Alan Moore

Suhas E Chelian

Rajiv Dayal

Alexander Ruben Stacey McCarthy

Jennifer Iglesias

Victor Puchkarev

Carlo Cruz

Aleix Paris I Bordas

Xinrui Wang

Aspects of the disclosure provide for controlling an autonomous vehicle. For instance, a trajectory for the autonomous vehicle to traverse in order to follow a route to a destination may be generated. A first error value for a boundary of an object, a second error value for a location of the autonomous vehicle, a third error value for a predicted future location of the object may be received. An uncertainty value for the object may be determined by combining the first error value, the second error value, and the third error value. A lateral gap threshold for the object may be determined based on the uncertainty value. The autonomous vehicle may be controlled in an autonomous driving mode based on the lateral gap threshold for the object.

Lateral gap planning for autonomous vehicles

A method performed by an autonomous device includes identifying a current movement of an operator based on monitoring the operator of the autonomous device. The method also includes inferring an intended direction of travel for the autonomous device based identifying the current movement. The method further includes identifying one or more objects in a current environment and limitations of the current environment. The method still further includes determining the action to be performed based on inferring the intended direction of travel and also identifying the one or more objects and the limitations of the current environment. The method also includes performing, by the autonomous device, the action.

Intent based control of a robotic device

A computing device configured to display a virtual representation of an environment of a robot includes a display device, a memory, and a processor coupled to the memory. The processor is configured to receive data from the one or more sensors of the robot with respect to an object within an environment of the robot. The processor is also configured to display a virtual representation of the object within a virtual mapping of the environment based on the data received from the one or more sensors. The processor is further configured to receive input data selecting the virtual representation of the object. The processor is also further configured to send instructions to the robot to act in response to the received input data.

Augmented reality overlay

A method includes providing a virtual representation of an environment of a robot, the virtual representation including an object representation of an object in the environment. The method further includes receiving manipulation input from a user to teleoperate the robot for manipulation of the object. The method also includes alerting the user to an alignment dimension based upon the manipulation input, receiving confirmation input from the user to engage the alignment dimension, and constraining at least one dimension of movement of the object according to the alignment dimension.

Systems and methods for dimensionally-restricted robotic teleoperation

A method for controlling a robotic device includes monitor an action of an operator of the robotic device. The method also includes inferring an intended target based on the monitored action. The method further includes determining an intended action for the intended target. The method still further includes controlling the robotic device to perform the intended action.

A teleoperation system includes a robot comprising an actuator configured to move at least a portion of the robot, and a remote computing device comprising: one or more processors, one or more sensors communicatively coupled to the one or more processors, a non-transitory memory component communicatively coupled to the one or more processors, and machine readable instructions stored in the non-transitory memory component. The remote computing device obtains information about a user proximate to the remote computing device, identifies the user based on the obtained information, obtains an action of a user, retrieves an individual profile for the user based on the identified user, determines an intended instruction related to a task based on the action of the user related to the task and the individual profile for the user, and instructs the robot to implement the task with the actuator based on the intended instruction.

Methods and systems for implementing customized motions based on individual profiles for identified users

A method includes providing a virtual representation of an environment of a robot. The virtual representation includes an object representation of an object in the environment. The method further includes determining an attribute of the object within the environment of the robot. The attribute includes at least one of occupancy data, force data, and deformation data pertaining to the object. The method further includes receiving a user command to control the robot to move with respect to the object, and modifying a received user command pertaining to the representation of the object based upon the attribute.

Systems and methods for conditional robotic teleoperation

A method includes detecting an object in a real environment of a robot. The method further includes inferring an expected property of the object based upon a representation of the object within a representation of the real environment of the robot. The method also includes sensing, via a sensor of the robot, a presently-detected property of the object in the real environment corresponding to the expected property. The method still further includes detecting a conflict between the expected property of the object and the presently-detected property of the object.

Systems and methods for na&#xef;ve physics for contact and contact-awareness in robotic teleoperation

A method for operating an arrival notification system at a location includes receiving a request to generate a notification from an arrival notification system. The method also includes receiving, in response to the request, a token. The method further includes enabling an output from a component of the arrival notification system when the token is validated.

Selective arrival notification system

A method includes presenting a virtual representation of an environment of a robot, receiving a first user command to control the robot within the environment, rendering a predicted version of the virtual representation during a period of latency in which current data pertaining to the environment of the robot is not available, updating the predicted version of the virtual representation based upon a second user command received during the period of latency, and upon conclusion of the period of latency, reconciling the predicted version of the virtual representation with current data pertaining to the environment of the robot.

Systems and methods for latency compensation in robotic teleoperation

Robot systems including a robot, an on-premise computing device, and a cloud services system are disclosed. An on-premise computing device includes a processor and a non-transitory memory device storing machine-readable instructions that, when executed by the processor, cause the processor to receive raw sensor data from a robot, pre-process the sensor data to remove unnecessary information from the sensor data and obtain pre-processed data, transmit the pre-processed data to a cloud services system, receive, from the cloud services system, an object type, an object pose, and an object location of an object corresponding to the pre-processed data, and transmit a signal to the robot based on at least one of the object type, the object pose, and the object location of the object.

Robot systems incorporating cloud services systems

A system for providing, to a user, navigation directions to a point of interest. The system includes a device having an output unit configured to output the navigation directions to the point of interest. The device is configured to communicate, to an unmanned vehicle, a point of interest identification. The system also includes the unmanned vehicle having a camera configured to detect image data. The unmanned vehicle is configured to determine whether the point of interest identification is detected from the image data. The unmanned vehicle is also configured to determine a point of interest location when the point of interest identification is detected from the image data. The unmanned vehicle is also configured to determine point of interest navigation data based on the point of interest location. The unmanned vehicle is also configured to communicate, to the device, the point of interest navigation data.

Systems and methods for providing environment information using an unmanned vehicle

Methods, systems, and apparatus for an automated mailbox system. The automated mailbox system includes a personal device configured to manage a robot. The robot includes a base that forms a foundation for the robot and is coupled to a post. The robot includes a mailbox having a door. The mailbox is configured to receive documents when in the open position. The robot includes a transportation component that moves the robot in multiple directions among multiple locations. The transportation component is configured to move the robot in a first direction to a first location.

Automated mailbox

A device for providing spatial information to a user. The device includes a camera configured to detect image data. The device also includes an accelerometer configured to determine step data. The device also includes a processor connected to the camera and the accelerometer and configured to determine a distance travelled per step of the user based on the image data and the step data. The processor is also configured to determine a distance to a reference point based on the image data. The processor is also configured to determine a number of steps corresponding to the distance to the reference point based on the distance travelled per step of the user. The device also includes an output unit connected to the processor and configured to output the spatial information indicating the number of steps corresponding to the distance to the reference point.

Growing community of inventors

San Jose, CA, United States of America

Allison Thackston