Baidu USA LLC

Baidu.com Times Technology (beijing) Co. Ltd.

Tiffany Zhang

Manjiang Zhang

Kwan Oh

Davy Huang

Xiangtao You

Oh Kwan

Xu Zhou

Min Guo

Shengjin Zhou

Ji Li

Shuai Wang

Yaoming Shen

Jeongho Lyu

Chang Shu

Dan N Retter

In one embodiment, a system for operating an autonomous driving vehicle (ADV) includes a number of modules. These modules include at least a perception module to perceive a driving environment surrounding the ADV and a planning module to plan a path to drive the ADV to navigate the driving environment. The system further includes a bus coupled to the modules and a sensor processing module communicatively coupled to the modules over the bus. The sensor processing module includes a bus interface coupled to the bus, a sensor interface to be coupled to a first set of one or more sensors mounted on the ADV, a message queue to store messages published by the sensors, and a message handler to manage the messages stored in the message queue. The messages may be subscribed by at least one of the modules to allow the modules to monitor operations of the sensors.

Open and safe monitoring system for autonomous driving platform

In one embodiment, a computer-implemented method of performing a secure boot operation in an autonomous driving vehicle includes reading a first marker from a storage device in which the storage device includes a plurality of partitions and at least the first marker. The plurality of partitions includes a first partition including stored software, the first marker associated with the first partition, and wherein the first marker includes a unique identifier and an authentication code. The method further includes determining if the read first marker associated with the first partition is valid during a boot-up operation and executing the stored software in the first partition if the read first marker is determined valid.

Boot failure recovery scheme for hardware-based system of autonomous driving vehicles

Systems and methods are disclosed for performing manufacturing testing on an autonomous driving vehicle (ADV) sensor board. A sensor unit of the ADV includes a plurality of sensor I/O channels that provide information to the ADV perception and planning module, to navigate the ADV. An array of sensors is emulated on a sensor unit test board. The sensor unit includes a small software that manages the flow of testing the sensor unit. The sensor unit test board provides emulated sensor data for, e.g., GPS, LIDAR, RADAR, inertial measurement, one or more cameras, humidity, temperature, and pressure, and throttle, braking, and steering inputs. Each emulated sensor includes its own data validity checker to ensure that each sensor I/O channel of the sensor unit is tested. The sensor unit test board can include an LED for each I/O channel that indicates a pass/fail status of the test for the I/O channel.

Flexible test board to improve sensor i/o coverage for autonomous driving platform

In one embodiment, a sensor unit to be utilized in an autonomous driving vehicle (ADV) includes a sensor interface that can be coupled to a number of sensors mounted on a number of different locations of the ADV. The sensor unit further includes a host interface that can be coupled to a host system such as a planning and control system utilized to autonomously drive the vehicle. The sensor unit further includes a number of data transfer modules corresponding to the sensors. Each of the data transfer modules can be configured to operate in one of the operating modes, dependent upon the type of the corresponding sensor. The operating modes include a low latency mode, a high bandwidth mode, and a memory mode.

Data transfer logic for transferring data between sensors and planning and control of an autonomous driving vehicle

The disclosure describes various embodiments of validating data synchronization between an active sensor and a passive sensor. According to an exemplary method of validating sensor synchronization between an active sensor and a passive sensor, a synchronization device receives a first signal from the active sensor, the first signal indicating that the active sensor has transmitted laser points to a measure board. In response to the first signal, the synchronization device transmits a second signal to the passive sensor to trigger the passive sensor to capture an image of the measure board. A synchronization validation application can perform an analysis of the image of the measure board in view of timing of the first signal and second signal to determine whether the passive sensor and the active sensor are synchronized with each other.

Sensor synchronization offline lab validation system

Diagnosing a sensor processing unit of an autonomous driving vehicle is described. An example computer-implemented method can include transmitting an executable image of a sensor processing application from a host system to the sensor processing unit via at least one of a universal asynchronous receiver-transmitter (UART) or an Ethernet connection. The method also includes causing the sensor processing unit to execute and launch the executable image of the sensor processing application in the DRAM from the eMMC storage device. The method also includes transmitting a sequence of predetermined commands to the executed sensor processing application to perform a plurality of sensor data processing operations on sensor data obtained from a plurality of sensors or sensor simulators associated with an autonomous driving vehicle. The method also includes comparing processing results of the sensor processing operations against expected processing results to determine whether the sensor processing application operates properly.

Automated factory testflow of processing unit with sensor integration for driving platform

A sensor unit includes a sensor interface, host interface, and pre-processing hardware. The sensor interface is coupled to a plurality of cameras configured to capture images around an autonomous driving vehicle (ADV). The host interface is coupled to a perception and planning system. The pre-processing hardware is coupled to the sensor interface to receive images from the plurality of cameras and to perform one or more pre-processing functions on the images and to transmit pre-processed images to the perception and planning system via the host interface. The perception and planning system is configured to perceive a driving environment surrounding the ADV based on the pre-processed images and to plan a path to control the ADV to navigate through the driving environment. The pre-processing functions can adjust for different calibrations and formats across the plurality of cameras.

Flexible hardware design for camera calibration and image pre-procesing in autonomous driving vehicles

In one embodiment, a sensor unit receives a first GPS message from a GPS sensor, where the sensor unit is coupled between sensors and a perception and planning system of an autonomous driving vehicle (ADV). The sensor unit determines a type of the first GPS message by matching a predetermined field of the first GPS message with a list of predetermined data patterns. Each of the predetermined data patterns corresponds to one of the predetermined types of GPS messages and decodes a payload of the first GPS message using a decoding algorithm associated with the type of the first GPS message.

Flexible GPS message decoder for decoding GPS messages during autonomous driving

In one embodiment, a system determines a difference in time between a local time source and a time of a GPS sensor. The system determines a max limit in difference and a max recovery increment or max recovery time interval for a smooth time source recovery. The system determines that the difference between the local time source and a time of the GPS sensor to be less than the max limit. The system plans a smooth recovery of the time source to converge the local time source to a time of the GPS sensor within the max recovery time interval. The system generates a timestamp based on the recovered time source to timestamp sensor data for a sensor unit of the ADV.

Time source recovery system for an autonomous driving vehicle

A sensor unit includes a sensor interface coupled to a number of sensors and a host interface coupled to a host system utilized to autonomously drive the vehicle. The sensor unit further includes sensor control modules corresponding to the sensors. Each sensor control module includes delay time control logic, delay adjustment logic, and a trigger signal generator. The delay time control logic is to receive a pulse time adjustment (PTA) value from the host system. The delay adjustment logic is to receive a trigger time adjustment (TTA) value from the host system. The delay adjustment logic is to modify timing of at least a portion of the pulses of a pulse signal based on the PTA value and the TTA value. The trigger signal generator is to generate a trigger signal based on the modified pulse signal and to transmit the trigger signal to a corresponding sensor.

Trigger logic to trigger sensors of an autonomous driving vehicle for capturing data

In one embodiment, a system receives, at a sensor unit, a global positioning system (GPS) pulse signal from a GPS sensor of the autonomous driving vehicle (ADV), where the GPS pulse signal is a RF signal transmitted by a satellite to the GPS sensor, where the sensor unit is coupled to a number of sensors mounted on the ADV to perceive a driving environment surrounding the ADV and to plan a path to autonomously drive the ADV. The system receives a first local oscillator signal from a local oscillator. The system synchronizes the first local oscillator signal to the GPS pulse signal in real-time, including modifying the first local oscillator signal based on the GPS pulse signal. The system generates a second oscillator signal based on the synchronized first local oscillator signal, where the second oscillator signal is used to provide a time to at least one of the sensors.

GPS based high precision timestamp generation circuit for an autonomous driving vehicle

A sensor unit used in an ADV includes a sensor interface that can be coupled to cameras mounted on the ADV. The sensor unit further includes a host interface that can be coupled to a host system. The host system is configured to perceive a driving environment surrounding the ADV based on at least image data obtained from the cameras and to plan a path to autonomously drive the ADV. The sensor unit further includes one or more data acquisition modules corresponding to the cameras. Each data acquisition module includes a pixel alignment module and a frame processing module. The pixel alignment module is configured to reformat pixels of image data from an original format associated with the corresponding camera to a predetermined format. The frame processing module is configured to generate an image frame based on the image data and to transmit the image frame to the host system.

Growing community of inventors

Sunnyvale, CA, United States of America

Tiffany Zhang