Guangzhou Xiaopeng Autopilot Technology Co., Ltd.

Cheng Lu

Chen Bai

Yanjun Zhu

Chengzhang Zhong

Runyu Mao

Yatong An

A system and a method for performing unified human motion prediction (HMP) are provided. In operation, the system provides a unified HMP module, which includes an encoder configured to encode a plurality of existing two-dimensional (2D) and three-dimensional (3D) pose sequences into a shared latent space V and a decoder configured to predict a plurality of predicted future 2D and 3D pose sequences based on the encoded shared latent space V, thus forming four pathways of knowledge flow. The system then trains the unified HMP module and the shared latent space V in a weakly supervised manner with input features X with actual future 2D and 3D pose sequences by minimizing a loss between the predicted future 2D and 3D pose sequences and the actual future 3D and 3D pose sequences. Once the training is complete, the unified HMP may be performed using the trained encoded shared latent space V.

System and method for performing unified human motion prediction (HMP) for 2D and 3D poses

A system for three-dimensional geometric guided student-teacher feature matching includes a multi-modal teacher model configured to determine feature matching between a pair of RGB-D images, each RGB-D image being a combination of a RGB image and its corresponding depth image; a mono-modal student model configured to determine feature matching from the pair of RGB images and the teacher model, the teacher model guiding the student model to learn RGB-induced depth information for the feature matching on both coarse and fine levels; a coarse-level knowledge transfer loss function for determining loss of transferring coarse-level matching knowledge from the teacher model to the student model; and a fine-level knowledge transfer loss function for determining loss of transferring fine-level matching knowledge from the teacher model to the student model, wherein the fine-level knowledge transfer loss function guides the student model to learn a fine-level prediction distribution with priority.

System for three-dimensional geometric guided student-teacher feature matching (3DG-STFM)

Aspects of online camera calibration for autonomous driving systems are described herein. The aspects may include a wheel odometer configured to measure a current speed of a moving vehicle, a speed monitor configured to determine that the current speed of the moving vehicle is high or low, and a camera calibrator configured to calibrate, while the vehicle is moving, at least one front camera of the moving vehicle according to a position of a vanishing point of two lane lines when the current speed is high. When the current speed is low and the vehicle is still moving, the camera calibrator may be further configured to calibrate multiple side and rear cameras of the moving vehicle according to a pose graph that includes relative poses of the multiple side and rear cameras.

Growing community of inventors

Guangdong, China

Cheng Lu