Baidu USA LLC

Baidu.com Times Technology (beijing) Co. Ltd.

Apollo Intelligent Driving Technology (beijing) Co., Ltd.

Apollo Autonomous Driving USA LLC

Manjiang Zhang

Tiffany Zhang

Shuai Wang

Davy Huang

Min Guo

Shengjin Zhou

Xiangtao You

Kwan Oh

Oh Kwan

Xu Zhou

Shuangcheng Guo

Ji Li

Yaoming Shen

Qiang Wang

Lingchang Li

Xianfei Li

Lei Wang

Zhen Chen

Ning Qu

Xiangfei Zhou

Ji Wan

Xing Li

Haofeng Kou

Weiyu Wendy Lu

Chongchong Li

Youling Zou

Fu-Kang Liao

Bernard Deadman

Congshi Huang

Jia Feng

Ran Zhang

Hassan Kianinejad

Yanyan Zhang

Huimeng Zheng

Ruichang Cheng

Hualei Lei

Zirui Huang

Jeongho Lyu

Chang Shu

Jian Sheng

Haidong Liu

Hongshun Shen

Zhou Xu

A sensor system for an autonomous driving vehicle (ADV) includes a sensor interface coupled to a plurality of sensors, a host interface coupled to a host system of the ADV, and a self-adaptive sensor transfer unit coupled between the sensor interface and the host interface. The self-adaptive sensor transfer unit includes a sensor monitor module, configured to monitor a data rate of sensor data received from a sensor, and a configuration control module, configured to: receive a target data rate from the host via the host interface; receive the monitored data rate of the sensor data; and control the data rate of the sensor data to be within a threshold of the target data rate.

Sensor data transfer with self adaptive configurations for autonomous driving vehicle

A system obtains a performance profile corresponding to times taken to perform an inferencing by a machine learning (ML) model using a different number of processing resources from a plurality of processing resources. The system determines one or more groupings of processing resources from the plurality of processing resources, each grouping includes one or more partitions. The system calculates performance speeds corresponding to each grouping based on the performance profile. The system determines a grouping having a best performance speed from the calculated performance speeds. The system partitions the processing resources based on the determined grouping to perform the inferencing.

System on chip parallel computing of ML services and applications for partitioning resources based on the inference times

An autonomous driving vehicle (ADV) includes a computer network that includes at least one wireless gateway. The ADV also includes a plurality of network components coupled to the computer network, the plurality of network components including a target device. The ADV also includes a processor, configured to receive, through the at least one wireless gateway, a firmware to be installed on the target device, and to direct the firmware to the target device through a first network path of the computer network. In response to detecting a failure to update the target device with the firmware, the processor directs the firmware to the target device through a second network path of the computer network.

Firmware over-the-air (FOTA) failure reduction for autonomous driving vehicle

A hardware unit of an ADV comprises an input port to directly receive data from one or more sensors perceiving a driving environment. The hardware unit is coupled with the one or more sensors to perform data processing of the data from one or more sensors. The hardware unit comprises a monitor unit to monitor a data rate of output data after the data processing. The hardware unit further comprises a self-adjustment unit to dynamically configure and adjust the data processing based on the data rate of output data. The hardware unit further comprises an output port to transfer the output data to an autonomous driving system (ADS) of the ADV to control the ADV to drive autonomously based on the output data.

Dynamic signal transfer configuration for driverless vehicle remote monitoring

A video system includes a first data gathering node configured to receive a plurality of image streams from a plurality of cameras, respectively. Each of the plurality of cameras captures an environment of an autonomous driving vehicle (ADV). The first data gathering node tags the plurality of image streams with metadata that identifies each of the plurality of image streams, and combines the plurality of image streams with the metadata to form a combined image stream. A second data gathering node is communicatively coupled to the first data gathering node and is to receive the combined image stream from the first data gathering node and output the combined image stream with a second combined image stream.

Scalable video compression accelerator for autonomous driving

In some implementations, a method is provided. The method includes determining a first set of data acquisition characteristics of a first sensor of an autonomous driving vehicle. The method also includes determining a second set of data acquisition characteristics of a second sensor of the autonomous driving vehicle. The method further includes synchronizing a first data acquisition time of the first sensor and a second data acquisition time of the second sensor, based on the first set of data acquisition characteristics and the second set of data acquisition characteristics. The first sensor obtains first sensor data at the first data acquisition time. The second sensor obtains second sensor data at the second data acquisition time.

Synchronizing sensors of autonomous driving vehicles

In one embodiment, an exemplary computer-implemented method of storing point cloud data in an autonomous driving vehicle can include the operations of receiving raw point cloud data from a LiDAR sensor mounted on the autonomous driving vehicle, the raw point cloud data representing cloud data points acquired in response to laser beams emitted at a given angle; retrieving configuration information of the LiDAR sensor, the configuration information including at least a number of laser lines of the LiDAR sensor. The method further includes the operations of constructing, based on the configuration information, a data structure that includes a data entry for each of the cloud data points, the data entry including multiple fields for storing attributes of the cloud data point, each field having a bit width determined based on the configuration information using a predetermined algorithm; and writing the cloud data points to a storage medium using the data structure.

Point cloud format optimized for LiDAR data storage based on device property

Embodiments relate to systems and methods to optimize quantization of tensors of an AI model. According to one embodiment, a system receives an AI model having one or more layers. The system receives a number of input data for offline inferencing and applies offline inferencing to the AI model based on the input data to generate offline data distributions for the AI model. The system quantizes one or more tensors of the AI model based on the offline data distributions to generate a low-bit representation AI model, where each layer of the AI model includes the one or more tensors, where the one or more tensors include the one or more tensors. In one embodiment, the system applies online inferencing using the low-bit representation AI model to generate online data distributions for a feature map, and quantizes a feature map tensor based on the online data distributions.

Optimization methods for quantization of neural network models

A method and system is disclosed for protecting neural network models by segmenting partitions of the models into segments of pre-configured memory size, hashing the segmented models, and concatenating the hash segments. The concatenated hash segment may be further hashed, encrypted, and stored with the neural network models as an executable loadable file (ELF) in memories external to the neural network prior to the use of the models by the neural network. The models may include model weights of the inference layers and metadata. The model weights and the metadata may be hashed as separate hash segments and concatenated. Segmenting the models into segments of pre-configured memory size and hashing the segmented models offline prior to the operation of the neural network enables rapid validation of the models when the models are used in the inference layers during online operation of the neural network.

Methods to protect neural network models

In some implementations, a method of verifying operation of a sensor is provided. The method includes causing a sensor to obtain sensor data at a first time, wherein the sensor obtains the sensor data by emitting waves towards a detector. The method also includes determining that the detector has detected the waves at a second time. The method further includes receiving the sensor data from the sensor at a third time. The method further includes verifying operation of the sensor based on at least one of the first time, the second time, or the third time.

Verifying timing of sensors used in autonomous driving vehicles

A data processing system includes a host system and one or more expansion devices coupled to the host system over a bus. The host system may include one or more processors and a memory storing instructions, which when executed, cause the processors to perform autonomous driving operations to drive an autonomous driving vehicle (ADV). Each expansion device includes a switch device and one or more processing modules coupled to the switch device. Each processing module can be configured to perform at least one of the autonomous driving operations offloaded from the host system. At least one of the processing modules can be configured as a client node to perform an action in response to an instruction received from the host system. Alternatively, it can be configured as a host node to distribute a task to another client node within the expansion device. This host node in the expansion device can further cooperate with the host system via a host-to-host connection.

Autonomous driving computing and storage expansion device with flexible host and client configuration

An ADV includes a method to combine data from multiple sensors. The method compresses video data from a camera to generate compressed video data. The compressed video data are segmented. The method time synchronizes each segment of the compressed video data with data from other sensors. The method then combines each segment of the compressed video data with the corresponding time-synchronized sensor data for the other sensors. In one embodiment, each segment of the compressed video data is independently decodable. In another embodiment, each segment of the compressed video data includes a compressed video unit that is prepended with a buffered portion of the compressed video data that immediately precede the compressed video unit. The length of the compressed video unit is smaller than the length of the independently decodable segment to offer finer granularity in time synchronizing the compressed video data with the other sensor data with a tradeoff.

Multiple sensor data storage with compressed video stream in autonomous driving vehicles

A method, apparatus, and system for timing synchronization between multiple computing nodes in an autonomous vehicle host system is disclosed. Timing of a first computing node of an autonomous vehicle host system is calibrated based on an external time source. A first timing message is transmitted from the first computing node to a second computing node of the autonomous vehicle host system via a first communication channel between the first computing node and the second computing node. Timing of the second computing node is calibrated based on the first timing message, wherein immediately subsequent to the calibration of timing of the second computing node, timing of the first computing node and of the second computing node is synchronized.

Time synchronization scheme between different computation nodes in autonomous driving system

In one embodiment, a computer-implemented method of performing a secure boot operation in an autonomous driving vehicle includes reading a first marker from a storage device in which the storage device includes a plurality of partitions and at least the first marker. The plurality of partitions includes a first partition including stored software, the first marker associated with the first partition, and wherein the first marker includes a unique identifier and an authentication code. The method further includes determining if the read first marker associated with the first partition is valid during a boot-up operation and executing the stored software in the first partition if the read first marker is determined valid.

Boot failure recovery scheme for hardware-based system of autonomous driving vehicles

Systems and methods are disclosed for performing manufacturing testing on an autonomous driving vehicle (ADV) sensor board. A sensor unit of the ADV includes a plurality of sensor I/O channels that provide information to the ADV perception and planning module, to navigate the ADV. An array of sensors is emulated on a sensor unit test board. The sensor unit includes a small software that manages the flow of testing the sensor unit. The sensor unit test board provides emulated sensor data for, e.g., GPS, LIDAR, RADAR, inertial measurement, one or more cameras, humidity, temperature, and pressure, and throttle, braking, and steering inputs. Each emulated sensor includes its own data validity checker to ensure that each sensor I/O channel of the sensor unit is tested. The sensor unit test board can include an LED for each I/O channel that indicates a pass/fail status of the test for the I/O channel.

Flexible test board to improve sensor i/o coverage for autonomous driving platform

Growing community of inventors

Sunnyvale, CA, United States of America

Manjiang Zhang