Baidu USA LLC

Apollo Intelligent Driving Technology (beijing) Co., Ltd.

Kecheng Xu

Jinghao Miao

Jiangtao Hu

Qi Luo

Hongyi Sun

Yajia Zhang

Jiaming Tao

Jinyun Zhou

Jiacheng Pan

Yu Wang

Shu Jiang

Zejun Lin

Wesley Reynolds

Jiaxuan Xu

Yifei Jiang

Xiangquan Xiao

Fan Zhu

Dong Li

Wei Wei Wang

Wei Wang

Yu Long Cao

Feng Quan Liu

Liangliang Zhang

Fan Yang

Jingao Wang

Liyun Li

Haoyang Fan

Weiman Lin

Siyang Yu

Youling Zou

Yu Cao

Shuo Huang

Wei-Man Lin

Feng Liu

In one embodiment, an emergency vehicle detection system can be provided in the ADV travelling on a road to detect the presence of an emergency vehicle in a surrounding environment of the ADV using both audio data and visual data. The emergency vehicle detection system can use a trained neutral network to independently generate a detection result from the audio data, and use another trained network to independently generate another detection result from the visual data. The emergency vehicle detection system can fuse the two detection results to determine the position and moving direction of the emergency vehicle. The ADV can take appropriate actions in response to the position and moving direction of the emergency vehicle.

Emergency vehicle audio and visual detection post fusion

Sound signals are received by one or more microphones disposed at an ADV. The sound signals are analyzed to extract a feature of a road on which the ADV is driving. A road condition of the road is determined based on the extracted feature. A path planning and speed planning is performed to generate a trajectory based on the road condition. The ADV is controlled to drive autonomously according to the generated trajectory.

Audio-based technique to sense and detect the road condition for autonomous driving vehicles

In one embodiment, a set of dynamic vehicle parameters of an ADV associated with a road location are determined based on outputs of an IMU of the ADV measured when the ADV is traveling through the road location. Whether the set of dynamic vehicle parameters satisfy one of a first set of criteria and a second set of criteria is determined. In response to the dynamic vehicle parameters satisfying the first set of criteria for a first predetermined quantity of times or satisfying the second set of criteria for a second predetermined quantity of times, the speed limit associated with the road location is adjusted within a limited range spanning from a minimum speed limit to a maximum speed limit. Operations of the ADV when the ADV subsequently travels through the road location are controlled based at least in part on the adjusted speed limit.

IMU feedback based HD map speed limit adjustment system

Sound signals that are not audible are generated by one or more speakers disposed in the vehicle. Reflected sound signals from a driver or a passenger of the vehicle that are not audible are received by one or more microphones disposed in the vehicle. A behavior-induced acoustic pattern is detected based on the reflected ultrasound signals. The behavior-induced acoustic pattern is analyzed to recognize a behavior of the driver or the passenger of the vehicle. A response or an alert is generated according to the recognized behavior of the driver or the passenger in the vehicle.

In-vehicle acoustic monitoring system for driver and passenger

An obstacle state evolution of a spatial position of a moving obstacle over a period of time is determined. A lane-obstacle relation evolution of the moving obstacle with each of one or more lanes near the moving obstacle over the period of time is further determined. An intended movement of the moving obstacle is predicted based on the obstacle state evolution and the lane-obstacle evolution. Thereafter, a trajectory of the ADV is planned to control the ADV to avoid a collision with the moving obstacle based on the predicted intended movement of the moving obstacle. The above process is iteratively performed for each of the moving obstacles detected within a predetermined proximity of the ADV.

Lane-attention: predicting vehicles' moving trajectories by learning their attention over lanes

In one embodiment, a control command is generated with an MPC controller, the MPC controller including a cost function with weights associated with cost terms of the cost function. The control command is applied to a dynamic model of an autonomous driving vehicle (ADV) to simulate behavior of the ADV. One or more of the weights are based on evaluation of the dynamic model in response to the control command, resulting in an adjusted cost function of the MPC controller. Another control command is generated with the MPC controller having the adjusted cost function. This second control command can be used to effect movement of the ADV.

Learning based controller for autonomous driving

Systems and methods for generating labelled audio data and onboard validation of the labelled audio data utilizing an autonomous driving vehicle (ADV) while the ADV is operating within a driving environment are disclosed. The method includes recording a sound emitted by an object within the driving environment of the ADV, and converting the recorded sound into audio samples. The method further includes labelling the audio samples, and refining the labelled audio samples to produce refined labelled audio data. The refined labelled audio data is utilized to subsequently train a machine learning algorithm to recognize a sound source during autonomous driving of the ADV. The method further includes generating a performance profile of the refined labelled audio data based at least on the audio samples, a position of the object, and a relative direction of the object. The position of the object and the relative direction of the object are determined by a perception system of the ADV.

Audio logging for model training and onboard validation utilizing autonomous driving vehicle

Described herein are various embodiments for identifying miscalibrations in LiDAR sensors installed on an ADV in real time while the ADV is in motion, and notifying a user that the LiDAR sensors needs to be recalibrated. An exemplary method includes calculating an initial number of overlapping cloud points between the cloud point data from a first LiDAR sensor and a second LiDAR sensor; and replacing a set of existing calibration parameters of the second LiDAR sensor with multiple sets of recalibration parameters to calculate multiple revised numbers of overlapping cloud points between the point cloud data. A potential miscalibration can be detected in the first LiDAR sensor or the second LiDAR sensor when the initial number of overlapping cloud points is smaller than at least one revised number of overlapping cloud points. The potential miscalibration can be verified by repeating the above operations multiple times with a time of time following each repetition.

Recalibration determination system for autonomous driving vehicles with multiple LiDAR sensors

In one embodiment, a computer-implemented method for optimizing a controller of an autonomous driving vehicle (ADV) includes obtaining several samples, each sample having a set of parameters, iteratively performing, until a predetermined condition is satisfied: determining, for each sample, a score according to a configuration of the controller based on the set of parameters of the sample, applying a machine learning model to the samples and corresponding scores to determine a mean function and a variance function, producing a new sample as a minimum of a function of the mean function and the variance function with respect to an input space of the set of parameters, adding the new sample to the several samples, and outputting the new sample as an optimal sample, where parameters of the optimal sample are utilized to configure the controller to autonomously drive the ADV.

Bayesian Global optimization-based parameter tuning for vehicle motion controllers

In one embodiment, an ADV is routed by executing a first driving scenario that is active. The first driving scenario is one of a plurality of driving scenario types, each driving scenario type being associated with one or more stages to be executed while a corresponding driving scenario type is active. Based on an environmental condition around the ADV, a second driving scenario is set as active. The ADV is routed by executing the second driving scenario. When the second driving scenario exits, execution of the first driving scenario resumes at the one or more stages of the first driving scenario that remains to be executed.

Nested scenarios in planning for autonomous driving vehicles

In one embodiment, an autonomous driving system of an ADV perceives a driving environment surrounding the ADV based on sensor data obtained from various sensors, including detecting one or more lanes and at least a moving obstacle or moving object. For each of the lanes identified, an NN lane feature encoder is applied to the lane information of the lane to extract a set of lane features. For a given moving obstacle, an NN obstacle feature encoder is applied to the obstacle information of the obstacle to extract a set of obstacle features. Thereafter, a lane selection predictive model is applied to the lane features of each lane and the obstacle features of the moving obstacle to predict which of the lanes the moving obstacle intends to select.

Neural network with lane aggregation for lane selection prediction of moving objects during autonomous driving

In one embodiment, a method determines a route from a first location of an autonomous driving vehicle (ADV) to a second location within an open space, the first location being a current location of the ADV. The method determines an objective function based on the route, the objective function having a set of costs for maneuvering the ADV from the first location to the second location. The method determines environmental conditions of the open space and uses the environmental conditions to determine a set of weights, each weight to be applied to a corresponding cost of the objective function. The method optimizes the objective function in view of one or more constraints, such that an output of the objective function reaches minimum while the one or more constraints are satisfied and generates a path trajectory with the optimized objective function to control the ADV autonomously according to the path trajectory.

Open space path planning using inverse reinforcement learning

A method, an apparatus, a storage medium, and an electronic device for testing dynamic parameter of vehicle are provided. The method for testing dynamic parameter of vehicle provided by the present disclosure includes: first obtaining a control parameter for an autonomous vehicle; then controlling the vehicle to travel automatically under a given environment according to the control parameter, detecting and recording traveling data of the vehicle; and at last determining a dynamic parameter of the vehicle according to the traveling data. According to the method for testing dynamic parameter provided by the present disclosure, the characteristic of automatic driving of an autonomous vehicle is utilized to achieve an automatic measurement of the dynamic parameter, thereby reducing cost for calibrating the vehicle and significantly improving safety during the test. Additionally, human error caused by manually driving during the test can be avoided effectively.

Method, apparatus, storage medium and electronic device for testing dynamic parameter of vehicle

According to various embodiments, systems, methods, and mediums for operating an autonomous driving vehicles (ADV) are described. The embodiments use a number of machine learning models to extract features individually from audio data and visual data captured by sensors mounted on the ADV, and then to fuse these extracted features to create a concatenated feature vectors. The concatenated feature vector is provided to a multiplayer perceptron (MLP) as input to generate a detection result related to the presence of an emergency vehicle in the surrounding environment. The detection result can be used by the ADV to take appropriate actions to comply with the local traffic rules.

Machine learning model to fuse emergency vehicle audio and visual detection

In one embodiment, a process is performed during controlling Autonomous Driving Vehicle (ADV). Microphone signals sense sounds in an environment of the ADV. The microphone signals are combined and filtered to form an audio signal having the sounds sensed in the environment of the ADV. A neural network is applied to the audio signal to detect a presence of an audio signature of an emergency vehicle siren. If the siren is detected, a change in the audio signature to make a determination as to whether the emergency vehicle siren is a) moving towards the ADV, or b) not moving towards the ADV. The ADV can make a driving decision, such as slowing down, stopping, and/or steering to a side, based on if the emergency vehicle siren is moving towards the ADV.

Growing community of inventors

Sunnyvale, CA, United States of America

Kecheng Xu