Google Inc.

X Development LLC

Toyota Research Institute, Inc.

James Joseph Kuffner

Ryan Michael Hickman

James Robert Bruce

Anthony Gerald Francis, Jr

Arshan Poursohi

Chaitanya Gharpure

Thor Lewis

Gill A Pratt

Peter Elving Anderson-Sprecher

James M Adler

Julian MacNeille Mason

Sumit Jain

Jared Stephen Russell

Damon Kohler

Rohit Ramesh Saboo

Adrien David Gaidon

Sudeep Pillai

Nikolaos Michalakis

Andrew Logan Harp

Won S Chun

Ken Conley

Anthony Gerald Francis

Example implementations may relate to a robotic system configured to provide feedback. In particular, the robotic system may determine a model of an environment in which the robotic system is operating. Based on this model, the robotic system may then determine one or more of a state or intended operation of the robotic system. Then, based one or more of the state or the intended operation, the robotic system may select one of one or more of the following to represent one or more of the state or the intended operation: visual feedback, auditory feedback, and one or more movements. Based on the selection, the robotic system may then engage in one or more of the visual feedback, the auditory feedback, and the one or more movements.

Robot to human feedback

Acquiring labeled data can be a significant bottleneck in the development of machine learning models that are accurate and efficient enough to enable safety-critical applications, such as automated driving. The process of labeling of driving logs can be automated. Unlabeled real-world driving logs, which include data captured by one or more vehicle sensors, can be automatically labeled to generate one or more labeled real-world driving logs. The automatic labeling can include analysis-by-synthesis on the unlabeled real-world driving logs to generate simulated driving logs, which can include reconstructed driving scenes or portions thereof. The automatic labeling can further include simulation-to-real automatic labeling on the simulated driving logs and the unlabeled real-world driving logs to generate one or more labeled real-world driving logs. The automatically labeled real-world driving logs can be stored in one or more data stores for subsequent training, validation, evaluation, and/or model management.

Auto-labeling of driving logs using analysis-by-synthesis and unsupervised domain adaptation

Providing user assistance in a vehicle includes identifying a driving behavior of the vehicle based on an evaluation of information about manual operation of the vehicle and information about an environment surrounding the vehicle, and issuing an alert to a user prompting the user to implement corrective manual operation. The user assistance further includes receiving a traffic behavior model that describes a predominating driving behavior of a like population of reference vehicles, and issuing the alert to a user prompting the user to implement corrective manual operation in response to identifying that the driving behavior of the vehicle does not match the predominating driving behavior of the like population of reference vehicles. Under the corrective manual operation, the driving behavior of the vehicle matches the predominating driving behavior of the like population of reference vehicles.

Providing user assistance in a vehicle based on traffic behavior models

Providing user assistance in a vehicle includes identifying a traffic behavior of an object in an environment surrounding the vehicle based on an evaluation of information about the environment surrounding the vehicle while the vehicle is in the midst of manual operation, and issuing an alert to a user prompting the user to implement defensive manual operation. The user assistance further includes receiving a traffic behavior model that describes a predominating traffic behavior of a like population of reference objects, and issuing the alert to a user prompting the user to implement defensive manual operation in response to identifying that the traffic behavior of the object does not match the predominating traffic behavior of the like population of reference objects. Under the defensive manual operation, the traffic behavior of the object is addressed.

Autonomous driving includes evaluating information about an environment surrounding a vehicle, generating, based on the evaluation of the information about the environment surrounding the vehicle, a driving plan for performing a driving maneuver, and operating vehicle systems in the vehicle to perform the driving maneuver according to the driving plan. The autonomous driving further includes receiving a traffic behavior model that describes a predominating driving behavior of a like population of reference vehicles. Under the driving plan, a driving behavior of the vehicle matches the predominating driving behavior of the like population of reference vehiclesAutonomous driving includes identifying a driving behavior of a vehicle based on an evaluation of information about manual operation of the vehicle and information about an environment surrounding the vehicle, and operating vehicle systems in the vehicle to perform a driving maneuver according to a driving plan for performing the driving maneuver. The autonomous driving further includes receiving a traffic behavior model that describes a predominating driving behavior of a like population of reference vehicles, and operating the vehicle systems to perform the driving maneuver according to the driving plan in response to identifying that the driving behavior of the vehicle does not match the predominating driving behavior of the like population of reference vehicles. Under the driving plan, the driving behavior of the vehicle matches the predominating driving behavior of the like population of reference vehicles.

Autonomous driving includes identifying a traffic behavior of an object in an environment surrounding a vehicle based on an evaluation of information about the environment surrounding the vehicle while the vehicle is in the midst of manual operation, and operating vehicle systems in the vehicle to perform a driving maneuver according to a driving plan for performing the driving maneuver. The autonomous driving further includes receiving a traffic behavior model that describes a predominating traffic behavior of a like population of reference objects, and operating the vehicle systems to perform the driving maneuver according to the driving plan in response to identifying that the traffic behavior of the object does not match the predominating traffic behavior of the like population of reference objects. Under the driving plan, the traffic behavior of the object is addressed.

Systems and methods for automatically re-energizing a vehicle. A vehicle determines a current energy source level in the vehicle's energy storage system. When the current energy source level is less than a threshold energy source level, a forthcoming location of the vehicle where the vehicle is non-operational is identified. Additionally, a window of time where the vehicle is located at the forthcoming location is identified. A mobile energy delivery (MED) vehicle is dispatched to the forthcoming location within the window of time to re-energize the vehicle.

Automatic re-energization of vehicles

Example methods and systems for augmented reality interfaces to display virtual representations of robotic device actions are provided. An example method includes receiving information that indicates an action or an intent of a robotic device to perform a task, and the action or the intent includes one or more of a planned trajectory of the robotic device to perform at least a portion of the task and an object to be handled by the robotic device to perform at least a portion of the task. The method also includes providing, for display by a computing device on an augmented reality interface, a virtual representation of the action or the intent, and the virtual representation includes as annotations on the augmented reality interface at least a portion of the planned trajectory of the robotic device or highlighting the object to be handled by the robotic device.

Methods and systems for augmented reality to display virtual representations of robotic device actions

Instructions indicative of changing a view of a virtual object may be received by a device. At least a portion of the virtual object may be viewable from a viewpoint that is at a given distance from a surface of the virtual object. The device may cause a change of the view along a rotational path around the virtual object in response to the receipt of the instructions based on the given distance being greater than a threshold distance. The device may cause a change of the view along a translational path indicative of a shape of the surface of the virtual object in response to the receipt of the instructions based on the given distance being less than the threshold distance.

Methods and systems for viewing a three-dimensional (3D) virtual object

This disclosure describes various embodiments for resource optimization in a vehicle. In an embodiment, a system for resource optimization in a vehicle is described. The system may comprise a memory; a processor coupled to the memory; and a resource optimization module. The resource optimization module may be configured to: monitor usage of local computing resources of the vehicle, the local computing resources comprising the processor and available bandwidth of a transmission medium; determine an availability of the local computing resources; evaluate data captured by one or more sensors of the vehicle; and determine whether to process the data locally or remotely based, at least in part, on the availability of the local computing resources and the data captured by the one or more sensors.

Resource optimization in vehicles

Systems, methods, and other embodiments described herein relate to acquiring embedded information from a roadway. In one embodiment, a method includes, in response to receiving a reflected signal resulting from a scanning signal interacting with the roadway, analyzing the reflected signal to detect a roadway signature embedded within the roadway. The method includes computing an identifier of the roadway signature as a function of features associated with the roadway signature that are embodied within the reflected signal. The method includes providing the embedded information about the roadway according to the identifier.

Systems and methods for roadway fingerprinting

System, methods, and other embodiments described herein relate to localizing a vehicle on a roadway. In one embodiment, a method includes, in response to detecting one or more indicators of a roadway signature within a reflected signal from the roadway, acquiring a fix on the roadway signature as a function of the one or more indicators that identify at least a segment of the roadway signature. The method includes localizing the vehicle on the roadway by correlating the roadway signature with a signature mapping that identifies a location for the segment of the roadway signature on the roadway.

Systems and methods for localizing a vehicle using a roadway signature

Methods and systems for proactively preventing hazardous or other situations in a robot-cloud interaction are provided. An example method includes receiving information associated with task logs for a plurality of robotic devices. The task logs may include information associated with tasks performed by the plurality of robotic devices. The method may also include a computing system determining information associated with hazardous situations based on the information associated with the task logs. For example, the hazardous situations may comprise situations associated with failures of one or more components of the plurality of robotic devices. According to the method, information associated with a contextual situation of a first robotic device may be determined, and when the information associated with the contextual situation is consistent with information associated with the one or more hazardous situations, an alert indicating a potential failure of the first robotic device may be provided.

Growing community of inventors

Mountain View, CA, United States of America

James Joseph Kuffner