Toyota Research Institute, Inc.

Toyota Jidosha Kabushiki Kaisha

Massachusetts Institute of Technology

Leland Stanford Junior University

Jonathan Decastro

Frank Permenter

Nikos Arechiga Gonzalez

Soonho Kong

Dennis Park

Soon Ho Kong

Guy Rosman

Nikos Arechiga

Nikos Arechiga-Gonzalez

Luke S Fletcher

Simon A I Stent

Tiffany L Chen

John J Leonard

Daniel J Brooks

Shabnam Hakimi

Emily Sarah Sumner

Deepak Edakkattil Gopinath

Daniel Stonier

Karen Y Leung

Daniel N Jackson

Sagar Behere

Dimitrios Koutentakis

Mike Meichang Wang

Feng Ping Angela Leong

Xin Zhang

Systems and methods for learning and managing robot user interfaces are disclosed herein. One embodiment generates, based on input data including information about past interactions of a particular user with a robot and with existing HMIs of the robot, a latent space using one or more encoder neural networks, wherein the latent space is a reduced-dimensionality representation of learned behavior and characteristics of the particular user, and uses the latent space as input to train a decoder neural network associated with (1) a new HMI distinct from the existing HMIs or (2) a particular HMI among the existing HMIs to alter operation of the particular HMI. The trained first decoder neural network is deployed in the robot to control, at least in part, operation of the new HMI or the particular HMI in accordance with the learned behavior and characteristics of the particular user.

Systems and methods for learning and managing robot user interfaces

Systems and methods for a powered, robotic exoskeleton, or exosuit, for a user's limbs and body are provided. The exosuit may be equipped with airbag devices mounted at various locations on the suit. The exosuit may include on-board computing equipment that can sense, compute control commands in real-time, and actuate limbs and airbags to restore stability (fall prevention) and minimize injuries due to falls, should they happen (fall protection).

Wearable exoskeleton

A method performed by an autonomous vehicle includes identifying a condition preventing the autonomous vehicle from proceeding along an intended route. The method also includes prompting a passenger of the autonomous vehicle to interact with a driver of a first vehicle in response to identifying the condition. The method further includes receiving, from the passenger, an input at an interface of the autonomous vehicle indicating a successful interaction or an unsuccessful interaction with the driver. The method also includes controlling the autonomous vehicle to proceed along the intended route based on the input indicating the successful interaction with the driver.

Actively adapting to driving environments based on human interactions

A method for certified control of a self-driving ego vehicle is described. The method includes analyzing a safety situation of the self-driving ego vehicle to determine a proposed vehicle control action using a main controller of the self-driving ego vehicle. The method also includes presenting, by the main controller, the proposed vehicle control action to an interlock controller, including a certificate of the proposed vehicle control action. The method further includes checking a safety certification evidence from the certificate by the interlock controller using a predefined safety argument to verify the safety certification evidence of the certificate. The method also includes directing, by a low-level controller, the self-driving ego vehicle to perform a certified vehicle control action.

Certified control for self-driving cars

A personalized notification method for a mobility as a service (MaaS) vehicle includes receiving conditional personalized notification features from a passenger of the MaaS vehicle. The method also includes monitoring current driving environment of the MaaS vehicle to determine whether a condition of the conditional personalized notification features is satisfied. The method further includes notifying the passenger when the condition is satisfied via at least one localized output device in a compartment of the MaaS vehicle.

Personalized notification system for mobility as a service

A driver monitor system and method for predicting impairment of a user of a vehicle. The system includes video cameras, an input device for inputting a list of medications being taken by the driver. Processing circuitry predicts side effects of the medications based on the half-life of the medication, detecting eye gaze movement, eye lid position, and facial expression of the user using images from the video camera, predicting whether the user is transitioning into an impaired physical state that is a side effect of the medications, verifying the side effect of the medications, determining whether the user is fit to drive using the verified side effects of the medications, and outputting to the vehicle an instruction to operate the vehicle in a level of automation that makes up for the at least one side effect or to perform a safe pull over operation of the vehicle.

Predictive impairment monitor system and method

Aspects of the disclosure provide a parking method and system for a vehicle. For example, the parking system can include a processor and an output circuit. As another example, the parking system can include the processor and a compelling unit. The processor can be configured to determine whether a first weather condition, such as snowing, raining, icing and hailing, at a first parking location meets a first criterion, and identify, when the first weather condition meets the first criterion, a second parking location at which a second weather condition meets a second criterion different from the first criterion. The output unit can be configured to output an alert to the vehicle parked at the first parking location indicating that the vehicle is to be moved to the second parking location. The compelling unit can be configured to compel the vehicle to move to the second parking location.

Predictive parking due to weather

A method for synthesizing parameters for control of a closed loop system based on a differentiable simulation model of the closed loop system includes determining requirements/specifications for the closed loop system in signal temporal logic (STL). The method also includes selecting a parametric control law having a differentiable parameter control function. The method also includes converting the requirements in signal temporal logic into differentiable computational graph. The method further includes building the differentiable simulation model as a differentiable computational graph. Furthermore, the method includes automatically learning values of parameters for the differentiable parameter control function of the closed loop system by backpropagating an error.

Simulation-based technique to synthesize controllers that satisfy signal temporal logic specifications

An artificial intelligence perception system for detecting one or more objects includes one or more processors, at least one sensor, and a memory device. The memory device includes an image capture module, an object identifying module, and a logical scaffold module. The image capture module and the object identifying module cause the one or more processors to obtain sensor information of a field of view from a sensor, identify an object within the sensor information, and determine at least one property of the object. The logical scaffold module causes the one or more processors to determine, by a logical scaffold, when the at least one property of the object as determined by the object identifying module is one of a true condition or a false condition.

System and method for detecting errors and improving reliability of perception systems using logical scaffolds

A method for controlling an autonomous vehicle includes navigating the autonomous vehicle based on a set of rules. The method also includes identifying an abnormality in a current driving situation. The method further includes prompting a passenger of the autonomous vehicle to interact with a driver of a first vehicle in response to identifying the abnormality. The method still further includes controlling the autonomous vehicle to violate one or more rules of the set of rules in response to an indication of a successful interaction with the driver.

System, methods, and other embodiments described herein relate to modeling dynamic agents in a surrounding environment of an ego vehicle. In one embodiment, a method includes, in response to receiving sensor data including present observations of a road agent of the dynamic agents in the surrounding environment, identifying previous observations of the road agent from an electronic data store. The method includes estimating a future state of the road agent using at least the present observations and the previous observations of the road agent to compute the future state according to a probabilistic model comprised of a transition model that accounts for dynamic behaviors of the road agent to characterize transitions between states, and an agent model that accounts for actions of the road agent. The method includes controlling one or more vehicle systems of the ego vehicle according to the future state of the road agent.

Systems and methods for agent tracking

System, methods, and other embodiments described herein relate to improving situational awareness of a user. In one embodiment, a method includes, in response to acquiring, in a personal device, sensor data from a set of sensors that perceive a surrounding environment of the user, identifying from the sensor data one or more objects in the surrounding environment relative to the user. The method includes determining a threat to the user from the one or more objects according to at least trajectories of the one or more objects embodied in the sensor data. The method includes generating, via the personal device, an indicator to inform the user of the one or more objects according to the threat.

Growing community of inventors

Arlington, MA, United States of America

Jonathan Decastro