Tusimple, Inc.

Yi Luo

Yi Wang

Ke Xu

Wentao Zhu

Xiaoling Han

Xue Mei

Panqu Wang

Bolun Zhang

Xiaodi Hou

Mingdong Wang

Chenzhe Qian

Systems and methods for detecting trailer angle are provided. In one aspect, an in-vehicle control system includes an optical sensor configured to be mounted on a tractor so as to face a trailer coupled to the tractor, the optical sensor further configured to generate optical data indicative of an angle formed between the trailer and the tractor. The system further includes a processor and a computer-readable memory in communication with the processor and having stored thereon computer-executable instructions to cause the processor to receive the optical data from the optical sensor, determine at least one candidate plane representative of a surface of the trailer visible in the optical data based on the optical data, and determine an angle between the trailer and the tractor based on the at least one candidate plane.

Systems and methods for detecting trailer angle

A system is disclosed including at least one memory including computer program instructions, which when executed by at least one processor, cause the system to at least generate, based on a plurality of images from a camera, a first map including a first plurality of features; generate, based on data from a light ranging sensor, a second map including a second plurality of features; and determine, based on a comparison of the first plurality of features and the second plurality of features, a position of the first map relative to the second map. A corresponding method and non-transitory computer-readable medium are also provided.

Feature matching and correspondence refinement and 3D submap position refinement system and method for centimeter precision localization using camera-based submap and LiDAR-based global map

Techniques are described for estimating pose of a camera located on a vehicle. An exemplary method of estimating camera pose includes obtaining, from a camera located on a vehicle, an image including a lane marker on a road on which the vehicle is driven, and estimating a pose of the camera such that the pose of the camera provides a best match according to a criterion between a first position of the lane marker determined from the image and a second position of the lane marker determined from a stored map of the road.

Camera pose estimation techniques

A method of localization for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs comprise instructions, which when executed by a computing device, cause the computing device to perform by one or more autonomous vehicle driving modules execution of processing of images from a camera and data from a LiDAR using the following steps comprising: computing, in response to features from a 3D submap and features from a global map, matching score between corresponding features of a same class between the 3D submap and the global map; selecting, for each feature in the 3D submap, a corresponding feature with the highest matching score from the global map; determining a feature correspondence to be invalid if a distance between corresponding features is larger than a threshold; and removing the invalid feature correspondence.

Feature matching and corresponding refinement and 3D submap position refinement system and method for centimeter precision localization using camera-based submap and LiDAR-based global map

A system for generating a ground truth dataset for motion planning is disclosed. The system includes sensors for data acquisition for motion planning of a vehicle, the sensors including a LiDAR; a calculating module configured to calculate LiDAR poses in response to data from the sensors and generate undistorted LiDAR scans; an extracting module configured to extract static points from a target undistorted LiDAR scan and generate a LiDAR static-scene point cloud; and a generating module configured to generate sparse image point correspondences for each pair of images, using the LiDAR static-scene point cloud.

Sensor calibration and time system for ground truth static scene sparse flow generation

A method of localization for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs comprise instructions, which when executed by a computing device, cause the computing device to perform by one or more autonomous vehicle driving modules execution of processing of images from a camera and data from a LiDAR using the following steps comprising: voxelizing a 3D submap and a global map into voxels; estimating distribution of 3D points within the voxels, using a probabilistic model; extracting features from the 3D submap and the global map; and classifying the extracted features into classes.

3D submap reconstruction system and method for centimeter precision localization using camera-based submap and LiDAR-based global map

A system for visual odometry is disclosed. The system includes: an internet server, comprising: an I/O port, configured to transmit and receive electrical signals to and from a client device; a memory; one or more processing units; and one or more programs stored in the memory and configured for execution by the one or more processing units, the one or more programs including instructions for: extracting representative features from a pair input images in a first convolution neural network (CNN) in a visual odometry model; merging, in a first merge module, outputs from the first CNN; decreasing feature map size in a second CNN; generating a first flow output for each layer in a first deconvolution neural network (DNN); merging, in a second merge module, outputs from the second CNN and the first DNN; generating a second flow output for each layer in a second DNN; and reducing accumulated errors in a recurrent neural network (RNN).

Neural network architecture system for deep odometry assisted by static scene optical flow

A method of visual odometry for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs includes instructions, which when executed by a computing device, causes the computing device to perform the following steps comprising: performing data alignment among sensors including a LiDAR, cameras and an IMU-GPS module; collecting image data and generating point clouds; processing, in the IMU-GPS module, a pair of consecutive images in the image data to recognize pixels corresponding to a same point in the point clouds; and establishing an optical flow for visual odometry.

Output of a neural network method for deep odometry assisted by static scene optical flow

A method of generating a ground truth dataset for motion planning is disclosed. The method includes performing data alignment, collecting data in an environment, using sensors, calculating, among other sensors, light detecting and ranging (LiDAR)'s poses, stitching multiple LiDAR scans to form a local map, refining positions in the local map based on a matching algorithm, and projecting 3D points in the local map onto corresponding images.

Sensor calibration and time method for ground truth static scene sparse flow generation

A method of generating a ground truth dataset for motion planning of a vehicle is disclosed. The method includes: obtaining undistorted LiDAR scans; identifying, for a pair of undistorted LiDAR scans, points belonging to a static object in an environment; aligning the close points based on pose estimates; and transforming a reference scan that is close in time to a target undistorted LiDAR scan so as to align the reference scan with the target undistorted LiDAR scan.

Undistorted raw LiDAR scans and static point extractions method for ground truth static scene sparse flow generation

A system for generating a ground truth dataset for motion planning of a vehicle is disclosed. The system includes an internet server that further includes an I/O port, configured to transmit and receive electrical signals to and from a client device; a memory; one or more processing units; and one or more programs stored in the memory and configured for execution by the one or more processing units, the one or more programs including instructions for: a corresponding module configured to correspond, for each pair of images, a first image of the pair to a LiDAR static-scene point cloud; and a computing module configured to compute a camera pose associated with the pair of images in the coordinate of the point cloud.

Growing community of inventors

San Diego, CA, United States of America

Yi Luo