Honda Motor Co., Ltd.

University of California

Osaka University

Keio University

Teruhisa Misu

Kumar Akash

Yi-Ting Chen

Ashish Tawari

Sujitha Catherine Martin

Xingwei Wu

Kentaro Yamada

Naoki Hosomi

Zhaobo K Zheng

Nanxiang Li

Ahmed Taha

Larry Davis

Hajime Yamada

Rakesh Gupta

John Canny

Ryu Takeda

Kazunori Komatani

Xitong Yang

Jinkyu Kim

Dario Pompili

Haibei Zhu

Wei Yang

Komei Sugiura

Vasili Ramanishka

Aasish Pappu

Ryusei Taniguchi

Hyungil Kim

Joseph L Gabbard

Xiaofeng Gao

Chun Ming Samson Ho

Chihiro Suga

Shumpei Hatanaka

Manisha Natarajan

Vidyasagar Sadhu

Fei Tao

Yuning Qiu

Jundi Liu

Kaihong Wang

A mobile object control device acquires a captured image obtained by capturing an image of surroundings of the mobile object by a camera mounted on a mobile object and an input directive sentence input by a user of the mobile object, inputs, when an image and a directive sentence are input, the captured image and the input directive sentence into a learned model learned to output one or more objects corresponding to the directive sentence in the image together with corresponding degrees of certainty to detect the one or more objects and the corresponding degrees of certainty, sequentially selects the one or more objects based on at least the degree of certainty and makes an inquiry to a user of the mobile object, and causes the mobile object to travel to an indicated position in the input directive sentence, which is specified based on a result of the inquiry.

Mobile object control device, mobile object control method, and storage medium

According to one aspect, systems, methods, and/or techniques associated with operator take-over prediction may include receiving a two-channel series of images of an operating environment through which a vehicle is travelling. A first series of images may be represented by labels corresponding to classified objects. A second series of images may be represented as a gaze heatmap or an eye tracking heatmap. Additionally, feeding encoded series of images through a three-dimensional (3D) convolutional neural network (CNN) to produce a first output, receiving sets of information corresponding to a first series of images and a second series of images in time, feeding sets of information through processing layers to produce additional outputs, concatenating the first output and the additional outputs to produce a concatenation output, and feeding the concatenation output through additional processing layers to generate an operator take-over prediction may be performed.

Operator take-over prediction

A control system comprises a travel control unit configured to cause a moving body to travel to a designated place designated by a dispatch request, in response to the dispatch request from a user. Interactive communication is performed between the user and the moving body via a mobile terminal held by the user, before the moving body reaches the designated place. Image data in which surroundings of the designated place are captured by an imaging unit is acquired, after the moving body has reached the designated place as a result of the interactive communication. Information that has been transmitted from the user in the interactive communication is evaluated, based on the image data that has been acquired.

Control system, control method, and storage medium for storing program to control the operation of a mobile device

A mobile object control device including a storage medium and a processor connected to the storage medium is presented. The processor acquires a photographed image, which is obtained by photographing surroundings of a mobile object by a camera mounted on the mobile object, and an input instruction sentence, which is input by a user of the mobile object; detects a stop position of the mobile object corresponding to the input instruction sentence in the photographed image by inputting at least the photographed image and the input instruction sentence into a trained model including a pre-trained visual-language model, the trained model being trained so as to receive input of at least an image and an instruction sentence to output a stop position of the mobile object corresponding to the instruction sentence in the image; and causes the mobile object to travel to the stop position.

Mobile object control device, mobile object control method, training device, training method, generation device, and storage medium

According to one aspect, systems and techniques for adaptive trust calibration may include usage of a driving style predictor, including a memory and a processor. The memory may store one or more instructions and the processor may execute one or more of the instructions stored on the memory to perform one or more acts, actions, or steps, such as receiving a current automated vehicle (AV) driving style, receiving an indication of an event and an associated event type, receiving an indication of a driver takeover, concatenating the current AV driving style and one or more of the event type or the driver takeover to generate an input, and passing the input through a neural network, which may include a gated recurrent unit (GRU), to generate a preference change associated with the AV driving style.

Adaptive trust calibration

According to one aspect, an adaptive driving style system may include a set of two or more sensors, a memory, and a processor. The set of two or more sensors may receive two or more sensor signals. The memory may store one or more instructions. The processor may execute one or more of the instructions stored on the memory to perform one or more acts, actions, or steps, including training a trust model using two or more of the sensor signals as input, training a preference model using the trust model and two or more of the sensor signals as input, and generating a driving style preference based on an adaptive driving style model including the trust model and the preference model.

Adaptive driving style

Aspects of adaptive trust calibration may include receiving a trust model for an occupant of an autonomous vehicle calculated based on occupant sensor data and a first scene context sensor data, and/or receiving a second scene context sensor data associated with an environment of the autonomous vehicle, determining an over trust scenario or an under trust scenario based on the trust model and a trust model threshold, and generating and implementing a human machine interface (HMI) action or a driving automation action based on the determination of the over trust scenario or the determination of the under trust scenario, and/or the second scene context sensor data.

A system and method for learning temporally consistent video synthesis using fake optical flow that include receiving data associated with a source video and a target video. The system and method also include processing image-to-image translation across domains of the source video and the target video and processing a synthesized temporally consistent video based on the image-to-image translation. The system and method further include training a neural network with data that is based on synthesizing of the source video and the target video.

System and method for learning temporally consistent video synthesis using fake optical flow

Systems and methods for clustering human trust dynamics are provided. In one embodiment, a computer implemented method for clustering human trust dynamics is provided. The computer implemented method includes receiving trust data for a plurality of participants interacting with one or more agents in an interaction. The computer implemented method also includes identifying a plurality of phases for the interaction. The computer implemented method further includes extracting features characterizing trust dynamics from the trust data for at least one interaction for each participant of the plurality participants. The at least one interaction is between the participant and an agent of the one or more agents. The computer implemented yet further includes assigning the features characterizing trust dynamics to a phase of the plurality of phases. The computer implemented method includes grouping a subset of the participant of the plurality of participants based on the on features characterizing trust dynamics.

Systems and methods for clustering human trust dynamics

A system and method for detecting a perceived level of driver discomfort in an automated vehicle that include receiving image data associated with a driving scene of an ego vehicle, dynamic data associated with an operation of the ego vehicle, and driver data associated with a driver of the ego vehicle during autonomous operation of the ego vehicle. The system and method also include analyzing the image data, the dynamic data, and the driver data and extracting features associated with a plurality of modalities. The system and method additionally include analyzing the extracted features and detecting the perceived level of driver discomfort. The system and method further include analyzing the perceived level of driver discomfort and detecting a probable driver takeover intent of the driver of the ego vehicle to takeover manual operation of the ego vehicle.

System and method for detecting a perceived level of driver discomfort in an automated vehicle

A system and method for providing an RNN-based human trust model that include receiving a plurality of inputs related to an autonomous operation of a vehicle and a driving scene of the vehicle and analyzing the plurality of inputs to determine automation variables and scene variables. The system and method also include outputting a short-term trust recurrent neural network state that captures an effect of the driver's experience with respect to an instantaneous vehicle maneuver and a long-term trust recurrent neural network state that captures the effect of the driver's experience with respect to the autonomous operation of the vehicle during a traffic scenario. The system and method further include predicting a take-over intent of the driver to take over control of the vehicle from an automated operation of the vehicle during the traffic scenario.

System and method for providing an RNN-based human trust model

A system including an acquisition unit configured to acquire, from a user via a communication device associated with the user, target object data including a feature of a target object selected by the user, an analysis unit configured to analyze whether the target object data that has been acquired by the acquisition unit includes, as the feature, at least one of data of a proper noun or data of a character string related to the target object, and whether the target object data includes data of a color related to the target object, and an estimation unit configured to estimate a distance from the target object to the user, based on an analysis result of the analysis unit, wherein the estimation unit estimates the distance from the target object to the user such that the distance from the target object to the user in a case where the target object data includes at least one of the data of the proper noun or the data of the character string is shorter than the distance from the target object to the user in a case where the target object data includes the data of the color is provided.

System, information processing apparatus, vehicle, and method

An adaptive trust calibration based autonomous vehicle may include vehicle systems, a system behavior controller, and a driving automation controller. The system behavior controller may generate a driving automation signal indicative of a desired autonomous driving adaptation. The driving automation controller may control the vehicle systems based on parameters including a desired velocity, current velocity of the autonomous vehicle, desired minimum gap distance between the autonomous vehicle and a detected object, current gap distance gap between the autonomous vehicle and a detected object, relative velocity of the detected object with respect to the autonomous vehicle, desired time headway, desired maximum acceleration, desired braking deceleration, and an exponent. The driving automation controller may receive the driving automation signal and implement the desired autonomous driving adaptation via the vehicle systems by adjusting the parameters based on a type of object associated with the detected object.

In some examples, one or more characteristics of one or more driving scenes may be obtained. Based at least on the one or more characteristics, one or more behaviors of a simulated driver may be simulated via a machine learning model. An operation associated with one or more advanced driving assistance system (ADAS) functions may be performed based at least on the simulated one or more behaviors.

Toward simulation of driver behavior in driving automation

A system and method for improving driver situation awareness prediction using human visual sensory and memory mechanism that includes receiving data associated with a driving scene of a vehicle and an eye gaze of a driver of the vehicle. The system and method also include analyzing the data and extracting features associated with objects located within the driving scene and determining a situational awareness score that is associated with a situational awareness of the driver with respect to each of the objects located within the driving scene. The system and method further include communicating control signals to electronically control at least one system of the vehicle based on the situational awareness score that is associated with each of the objects located within the driving scene.

Growing community of inventors

Mountain View, CA, United States of America

Teruhisa Misu