Toyota Motor Engineering & Manufacturing North America, Inc.

Toyota Jidosha Kabushiki Kaisha

Other

Kyungtae Han

Ziran Wang

Prashant Tiwari

Rohit Gupta

Yanbing Wang

BaekGyu Kim

Yongkang Liu

Zhenyu Shou

Amr Abdelraouf

Rui Guo

Ahmed Hamdi Sakr

Hiromitsu Kobayashi

Dawei Chen

Sergei S Avedisov

Zhouqiao Zhao

Onur Altintas

Roger D Melen

Katsumi Nagata

Hongsheng Lu

John K Lenneman

Rohit Kumar Gupta

Miles J Johnson

Haoxin Wang

Akila C Ganlath

Paul Li

Xiaowen Jiang

Qi Chen

Chase Violetta

Jianyu Su

Nejib Ammar

Toshinori Esaka

Satoshi Nagashima

Yitao Chen

Hazem Abdelkawy

Runjia Du

Daniel Doe

A method for video instance segmentation is provided. The method includes inputting a plurality of video frames collected by a sensor of a vehicle to a trained machine learning model to obtain an n-th output from an n-th layer of the trained machine learning model and an n+1-st output from an n+1-st layer of the trained machine learning model, the trained machine learning model comprising a deep learning model and early-exit subnets, and in response to determining that a difference between the n-th output and the n+1-st output is less than a threshold value, controlling the vehicle based on the n+1-st output, the n+1-st output includes information about instances in the plurality of video frames.

Systems and methods for efficient video instance segmentation for vehicles using edge computing

A method may include learning reward functions for a plurality of first vehicles based on first vehicle data associated with the plurality of first vehicles using inverse reinforcement learning, associating each vehicle of the plurality of first vehicles with one cluster among a plurality of clusters based on the reward functions, determining a centroid reward function for each of the clusters based on the reward functions associated with each cluster, performing a comparison between second vehicle data associated with a second vehicle and the first vehicle data, determining a vehicle among the plurality of first vehicles having associated first vehicle data that is most similar to the second vehicle data based on the comparison, associating the second vehicle with the cluster associated with the determined vehicle, and controlling operation of the second vehicle based on the centroid reward function of the cluster associated with the second vehicle.

Method and system for modeling personalized car-following driving styles with model-free inverse reinforcement learning

Systems and methods are provided for learning a driving behavior during a cut-in cut-out event or other triggering event to create a profile that adjusts operation of the adaptive cruise control (ACC) component of the vehicle to mimic the preferences of the driver. The profile may be based on data collected during a previous cut-in cut-out event or other triggering event. The data may be transmitted to an adaptive cruise control system that uses the data as input to a machine learning model. Output of the machine learning model may update the profile for the driver that operates the vehicle in ACC during the cut-in cut-out event. When the vehicle is operating in ACC and an event is within a threshold value of the cut-in cut-out event occurs at a later time, the vehicle may apply rules defined in the profile.

Trigger event personalized adaptive cruise control (P-ACC)

An example operation includes one or more of receiving position data of an ego vehicle and of one or more surrounding vehicles of the ego vehicle via sensors of the ego vehicle while the ego vehicle and the surrounding vehicles are travelling along a road, predicting, via a machine learning model, a trajectory of the ego vehicle and trajectories of the one or more surrounding vehicles based on the position data, generating a routing instruction for the ego vehicle based on the predicted trajectory of the ego vehicle and the predicted trajectories of the one or more surrounding vehicles, and transmitting the routing instruction to a display associated with the ego vehicle.

Lane change assist for inexperienced driver

Embodiments of methods and systems include determining a driving maneuver to be performed by an ego vehicle based on a road geometry and a lane geometry, the ego vehicle surrounded by one or more surrounding vehicles, retrieving a trajectory model corresponding to the driving maneuver, encoding kinematic data of the surrounding vehicles relative to the ego vehicle to generate spatial relationships between the surrounding vehicles and the ego vehicle, predicting future trajectories of the surrounding vehicles based on the spatial relationships, the kinematic data, the road geometry, and the lane geometry, generating a collision coefficient based on the future trajectories and the driving maneuver, and when the collision coefficient is greater than a predetermined threshold value, determining an escape route for the ego vehicle, and causing the ego vehicle to autonomously operate based on the escape route.

Methods and systems for collision control of moving vehicles surrounded by vehicles

An active road surface maintenance system and method developed for connected vehicles with the aid of a mobility digital twin (MDT) framework. A method performed in a cloud-based digital space includes receiving data regarding a physical object from a physical space connected to a vehicle. The method also includes processing the data using machine learning to model road surface conditions, in which respective penalty values are assigned to corresponding road surfaces, a respective penalty value being higher the lower a condition of the corresponding road surface. The method also includes deriving instructions based on the modeled road surface conditions and the respective penalty values to guide actuation of the vehicle along a trajectory. The method further includes transmitting the instructions to the physical space connected to the vehicle to guide actuation of the vehicle.

Systems and methods for active road surface maintenance with cloud-based mobility digital twin

Systems and methods for selecting an edge node include a processor operable to send a request of bidding to edge nodes regarding performing a task for an inquiry vehicle, receive one or more bids from one or more bidding edge nodes of the edge nodes, wherein each bid includes a resource allocation offer and a price, calculate a preference value based on a weighted sum of preference factors for each bidding edge node, wherein the preference factors are determined based on the resource allocation offer, and the weight assigned to each preference factor is determined based on the task, select a performing edge node based on the preference values and the prices, and operate the inquiry vehicle based on a result of the task performed by the performing edge node.

Systems and methods for selecting edge nodes for task performance

System and method for reducing latency and bandwidth usage in reality devices include a reality device and one or more processors. The reality device includes a camera to operably capture a set of consequent frames of views external to a vehicle. The one or more processors are operable to monitor movements of one or more eyes of a user to obtain eye-tracking data using the reality device, identify an area of interest (AoI) in the frame based on the eye-tracking data, AoI history, or driving statistics of the user, crop the frame to obtain a reduced-size frame including the AoI in the frame, and transmit the reduced-size frame to an edge server for performing a task on behalf of the vehicle.

Systems and methods for resource-optimized mixed reality using user-centric adaptive object detection

An example operation includes one or more of receiving driving data of a first vehicle, receiving driving data of a second vehicle driving in a different lane than the first vehicle, determining that the first vehicle and the second vehicle will collide via execution of a machine learning model on the driving data of the first and second vehicles, and displaying a warning on a user interface associated with one or more of the first and second vehicles.

Omnidirectional collision avoidance

A speed limiter for a vehicle can be personalized. The speed limiter can be caused to be activated. A user can provide a user input on a user interface. The user input can include a speed limiter personalization setting. In response to the user input, the speed limiter personalization setting can be caused to be implemented.

Personalized speed limiter

A system for communication-aware federated learning includes a server and edge nodes. Each of the edge nodes trains a machine learning model using first local data obtained by sensors of corresponding edge node. Each of the edge nodes obtains network bandwidth for a channel between corresponding edge node and the server. One or more of the edge nodes determines a level of compression based on the bandwidth for the channel, compresses the trained machine leaning model based on the determined level of compression, and transmits the compressed trained machine learning model to the server. The server decompresses the compressed trained machine learning models and aggregates the decompressed trained machine learning models to obtain the aggregated machine learning model, and transmits the aggregated machine learning model to each of the edge nodes. Each of the edge nodes receives the aggregated machine learning model from the server.

Systems and methods for communication-aware federated learning

The disclosure includes embodiments for a medic system to respond to a medical condition of a vehicle occupant. A method according to some embodiments is executed by a processor. The method also includes determining, by the processor, that a driver of an ego vehicle is experiencing a debilitating medical condition. The method also includes overriding a protocol to decrease an autonomy level of the ego vehicle responsive to inattentiveness of the driver to a driving interface of the ego vehicle so that the driver can be inattentive to the driving interface and the autonomy level is not decreased. The method also includes modifying an operation of an autonomous driving system of the ego vehicle to increase the autonomy level of the ego vehicle to decrease a driving responsibility of the driver responsive to the debilitating medical condition.

Medical emergency detection in-vehicle caretaker

A method may include receiving training data comprising a time series of gaps between an ego vehicle and one or more lead vehicles at a plurality of time steps, embedding the training data into a fixed-length sequence, inputting the fixed-length sequence into a Transformer-RNN model comprising a Transformer component and an RNN component, wherein the transformer component applies attention to each data point of the fixed-length sequence based on a fixed number of previous inputs, and training the Transformer-RNN model, using the training data, to output a predicted gap at a future time step based on an input sequence of gaps.

Method and system for personalized car following with transformers and RNNs

The disclosure includes embodiments for identifying a transmitter of a Vehicle-to-Everything (V2X) message. In some embodiments, a method for an ego vehicle includes modifying an operation of a communication unit of the ego vehicle to receive a V2X message that includes identification data of a transmitter of the V2X message. The method includes executing a proactive vehicle control operation on the ego vehicle to modify a distance between the ego vehicle and a preceding vehicle ahead of the ego vehicle so that the distance satisfies a distance threshold. The method includes determining whether the preceding vehicle is the transmitter based on the identification data so that a reliable determination is achieved to improve a driving safety of the ego vehicle responsive to the distance satisfying the distance threshold.

Proactive message transmitter identification system

A method comprises determining a vectorized representation of positions of road agents and road geometry based on sensor data from a vehicle, inputting the vectorized representation of the positions of the road agents and the road geometry into a trained transformer network, predicting one or more road agent trajectories at one or more future time steps based on an output of the transformer network, predicting an acceleration of the vehicle at the one or more future time steps based on the predicted one or more road agent trajectories at the one or more future time steps, and causing the vehicle to perform the predicted acceleration at the one or more future time steps.

Growing community of inventors

Palo Alto, CA, United States of America

Kyungtae Han