Baidu USA LLC

Baidu.com Times Technology (beijing) Co. Ltd.

Apollo Intelligent Driving Technology (beijing) Co., Ltd.

Baidu.com Times Technology (beijing) Co., Ltd.

Qi Luo

Jiangtao Hu

Jinghao Miao

Shu Jiang

Yu Wang

Jinyun Zhou

Jiaxuan Xu

Jingao Wang

Qi Kong

Kecheng Xu

Jiaming Tao

Fan Zhu

Guang Yu Yang

Xiang Yu

Sen Hu

Runxin He

Liangliang Zhang

Yifei Jiang

Zhenguang Zhu

Yu Long Cao

Yuchang Pan

Dong Li

Yajia Zhang

Hongyi Sun

Haoyang Fan

Weiman Lin

Xiangquan Xiao

Weicheng Zhu

Wesley Reynolds

Zejun Lin

Jiarui He

Shiyu Song

Li Zhuang

Wenli Yang

Zhongpu Xia

Wei Wei Wang

Xiaoxin Fu

Hongye Li

Chunming Zhao

Longtao Lin

Wei Wang

Feng Quan Liu

Lin Ma

Chao Ma

Jiacheng Pan

Kuang Hu

Siyang Yu

Youling Zou

Wenda Zeng

Zongbao Feng

Shuo Huang

Chuming Zhao

Wei-Man Lin

Yu Cao

Feng Liu

Guang Yang

In one embodiment, an emergency vehicle detection system can be provided in the ADV travelling on a road to detect the presence of an emergency vehicle in a surrounding environment of the ADV using both audio data and visual data. The emergency vehicle detection system can use a trained neutral network to independently generate a detection result from the audio data, and use another trained network to independently generate another detection result from the visual data. The emergency vehicle detection system can fuse the two detection results to determine the position and moving direction of the emergency vehicle. The ADV can take appropriate actions in response to the position and moving direction of the emergency vehicle.

Emergency vehicle audio and visual detection post fusion

In one embodiment, an exemplary method includes the operations of receiving, at a profiling application, a record file recorded by the ADV for a driving scenario in an area, and a high definition map matching the area; extracting planning messages and perception messages from the record file; and aligning the planning message and the perception messages based on their timestamps. The method further includes calculating an individual performance score for each planning cycle of the ADV for the driving scenario based on the planning messages; calculating a weight for each planning cycle based on the perception messages and the high definition map; and then calculating a weighted score for the driving scenario based on individual performance scores and their corresponding weights.

Weighted planning trajectory profiling method for autonomous vehicle

Systems, methods, and media for factoring localization uncertainty of an ADV into its planning and control process to increase the safety of the ADV. The uncertainty of the localization can be caused by sensor inaccuracy, map matching algorithm inaccuracy, and/or speed uncertainty. The localization uncertainty can have negative impact on trajectory planning and vehicle control. Embodiments described herein are intended to increase the safety of the ADV by considering localization uncertainty in trajectory planning and vehicle control. An exemplary method includes determining a confidence region for an ADV that is automatically driving on a road segment based on localization uncertainty and speed uncertainty; determining that an object is within the confidence region, and a probability of collision with the ADV based on a distance of the object to the ADV; and planning a trajectory based on the probability of collision, and controlling the ADV based on the probability of collision.

Control and planning with localization uncertainty

Disclosed are performance metrics for evaluating the accuracy of a dynamic model in predicting the trajectory of ADV when simulating the behavior of the ADV under the control commands. The performance metrics may indicate the degree of similarity between the predicted trajectory of the dynamic model and the actual trajectory of the vehicle when applied with identical control commands. The performance metrics measure deviations of the predicted trajectory of the dynamic model from the actual trajectory based on the ground truths. The performance metrics may include cumulative or mean absolute trajectory error, end-pose difference (ED), two-sigma defect rate (ε), the Hausdirff Distance (HAU), the longest common sub-sequence error (LCSS), or dynamic time warping (DTW). The two-sigma defect rate represents the ratio of the number of points with true location error falling out of the 2σ range of the predicted location error over the total number of points in the trajectory.

Dynamic model evaluation package for autonomous driving vehicles

According to various embodiments, systems, methods, and media for evaluating an open space planner in an autonomous vehicle are disclosed. In one embodiment, an exemplary method includes receiving, at a profiling application, a record file recorded by the ADV while driving in an open space using the open space planner, and a configuration file specifying parameters of the ADV; extracting planning messages and prediction messages from the record file, each extracted message being associated with the open space planner. The method further includes generating features from the planning message and the prediction messages in view of the specified parameters of the ADV; and calculating statistical metrics from the features. The statistical metrics are then provided to an automatic tuning framework for tuning the open space planner.

Open space planner profiling tool for autonomous vehicle

Sound signals are received by one or more microphones disposed at an ADV. The sound signals are analyzed to extract a feature of a road on which the ADV is driving. A road condition of the road is determined based on the extracted feature. A path planning and speed planning is performed to generate a trajectory based on the road condition. The ADV is controlled to drive autonomously according to the generated trajectory.

Audio-based technique to sense and detect the road condition for autonomous driving vehicles

Sound signals that are not audible are generated by one or more speakers disposed in the vehicle. Reflected sound signals from a driver or a passenger of the vehicle that are not audible are received by one or more microphones disposed in the vehicle. A behavior-induced acoustic pattern is detected based on the reflected ultrasound signals. The behavior-induced acoustic pattern is analyzed to recognize a behavior of the driver or the passenger of the vehicle. A response or an alert is generated according to the recognized behavior of the driver or the passenger in the vehicle.

In-vehicle acoustic monitoring system for driver and passenger

A method of navigating an autonomous driving vehicle (ADV) includes determining a target function for an open space model based on one or more obstacles and map information within a proximity of the ADV, then iteratively performing first and second quadratic programming (QP) optimizations on the target function. Then, generating a second trajectory based on results of the first and second QP optimizations to control the ADV autonomously using the second trajectory. The first QP optimization is based on fixing a first set of variables of the target function. The second QP optimization is based on maximizing a sum of the distances from the ADV to each of the obstacles over a plurality of points of the first trajectory, and minimizing a difference between a target end-state of the ADV and a determined final state of the ADV using the first trajectory.

End dynamics and constraints relaxation algorithm on optimizing an open space trajectory

In one embodiment, a control command is generated with an MPC controller, the MPC controller including a cost function with weights associated with cost terms of the cost function. The control command is applied to a dynamic model of an autonomous driving vehicle (ADV) to simulate behavior of the ADV. One or more of the weights are based on evaluation of the dynamic model in response to the control command, resulting in an adjusted cost function of the MPC controller. Another control command is generated with the MPC controller having the adjusted cost function. This second control command can be used to effect movement of the ADV.

Learning based controller for autonomous driving

A simulation of an autonomous driving vehicle (ADV) includes capturing first data that includes a control command output by an autonomous vehicle controller of the ADV, and capturing second data that includes the control command being implemented at a control unit of the ADV. The control command, for example, a steering command, a braking command, or a throttle command, is implemented by the ADV to affect movement of the ADV. A latency model is determined based on comparing the first data with the second data, where the latency model defines time delay and/or amplitude difference between the first data and the second data. The latency model is applied in a virtual driving environment.

Dynamic model with actuation latency

In one embodiment, control of an autonomous driving vehicle (ADV) includes determining a current scenario of the ADV. Based on the scenario, a control algorithm is selected among a plurality of distinct control algorithms as the active control algorithm. One or more control commands are generated using the active control algorithm, based one or more target inputs. The control commands are applied to effect movement of the ADV.

Scenario based control of autonomous driving vehicle

Systems and methods for generating labelled audio data and onboard validation of the labelled audio data utilizing an autonomous driving vehicle (ADV) while the ADV is operating within a driving environment are disclosed. The method includes recording a sound emitted by an object within the driving environment of the ADV, and converting the recorded sound into audio samples. The method further includes labelling the audio samples, and refining the labelled audio samples to produce refined labelled audio data. The refined labelled audio data is utilized to subsequently train a machine learning algorithm to recognize a sound source during autonomous driving of the ADV. The method further includes generating a performance profile of the refined labelled audio data based at least on the audio samples, a position of the object, and a relative direction of the object. The position of the object and the relative direction of the object are determined by a perception system of the ADV.

Audio logging for model training and onboard validation utilizing autonomous driving vehicle

Systems and methods are disclosed for optimizing values of a set of tunable parameters of an autonomous driving vehicle (ADV). The controllers can be a linear quadratic regular, a 'bicycle model,' a model-reference adaptive controller (MRAC) that reduces actuation latency in control subsystems such as steering, braking, and throttle, or other controller ('controllers'). An optimizer selects a set tunable parameters for the controllers. A task distribution system pairs each set of parameters with each of a plurality of simulated driving scenarios, and dispatches a task to the simulator to perform the simulation with the set of parameters. Each simulation is scored. A weighted score is generated from the simulation. The optimizer uses the weighted score as a target objective for a next iteration of the optimizer, for a fixed number of iterations. A physical real-world ADV is navigated using the optimized set of parameters for the controllers in the ADV.

Automatic parameter tuning framework for controllers used in autonomous driving vehicles

In one embodiment, a computer-implemented method of operating an autonomous driving vehicle (ADV) includes perceiving a driving environment surrounding the ADV based on sensor data obtained from one or more sensors mounted on the ADV, determining a driving scenario, in response to a driving decision based on the driving environment, applying a predetermined machine-learning model to data representing the driving environment and the driving scenario to generate a set of one or more driving parameters, and planning a trajectory to navigate the ADV using the set of the driving parameters according to the driving scenario through the driving environment.

Neural network approach for parameter learning to speed up planning for complex driving scenarios

In one embodiment, a method of training dynamic models for autonomous driving vehicles includes the operations of receiving a first set of training data from a training data source, the first set of training data representing driving statistics for a first set of features; training a dynamic model based on the first set of training data for the first set of features; determining a second set of features as a subset of the first set of features based on evaluating the dynamic model, each of the second set of features representing a feature whose performance score is below a predetermined threshold. The method further includes the following operations for each of the second set of features: retrieving a second set of training data associated with the corresponding feature of the second set of features, and retraining the dynamic model using the second set of training data.

Automated training data extraction method for dynamic models for autonomous driving vehicles

Growing community of inventors

Sunnyvale, CA, United States of America

Qi Luo