Tsinghua University

Lenovo (beijing) Limited

Beijing University of Chemical Tehcnology

Huaping Liu

Xinyu Zhang

Jun Li

Zhiwei Li

Xingang Wu

Li Wang

Qifan Tan

Bin Fang

Yuchao Liu

Fuchun Sun

Jianxi Luo

Hui Li

Wei Zhang

Wei Xie

Mo Zhou

Tianyu Shen

Kunfeng Wang

Zhenhong Zou

Wenju Gao

Kangyao Huang

Zhiyu Zhang

Xiaoming Xie

Fengli Lu

Yiqian Lu

Shichun Guo

A method and an apparatus for collaborative end-to-end large model-oriented self-driving trajectory decision-making are provided. The method includes: processing an RGB image from ego vehicle, an RGB image from surrounding vehicle, and an RGB image from road-side by using a first feature extraction network to obtain a first image feature, a second image feature, and a third image feature, respectively; fusing first image feature, second image feature, and third image feature to obtain an image fusion feature; processing point cloud data of road-side to obtain a road-side point cloud feature; processing image fusion feature and road-side point cloud feature to obtain a first BEV feature and a second BEV feature; fusing first BEV feature and second BEV feature to obtain a fused BEV feature; and fusing prompt information and fused BEV feature to obtain text information, and then processing text information to obtain an ego vehicle trajectory decision-making result.

Method and apparatus for collaborative end-to-end large model-oriented self-driving trajectory decision-making

The present invention proposes a fabric defect detection method based on multi-modal deep learning. First, a tactile sensor is placed onto the fabric surface with different defects to collect the fabric texture images, a camera is used to collect the corresponding fabric external images, and a fabric external image and a fabric texture image constitute a set of fabric detection data; then, a feature extraction network and a multi-modal fusion network are connected to establish a classification model based on multi-modal deep learning, which uses the fabric texture image and fabric external image in each set of collected fabric detection data as input, and the fabric defect as output; said classification model is trained using the collected fabric detection data; finally, the trained classification model is used to detect the fabric defect. The present invention employs vision-touch complementary information, which can greatly improve the accuracy and robustness of detection.

Fabric defect detection method based on multi-modal deep learning

The invention proposes an automatic throat swab sampling system, which comprises an automatic to-be-tested person identity information checking and collection prompting unit, a to-be-tested person head positioning unit, a navigation positioning unit which enters the oral cavity of a to-be-detected person along with the tail end of the sample collection execution unit for collecting a throat image and determining depth information between a depth sensor and the throat of the to-be-detected person, a sample collection execution unit including a three-degree-of-freedom guide rail type device and a one-degree-of-freedom end execution mechanism, an automatic throat swab loading and unloading unit and a remote monitoring unit which are arranged on a working platform. In a throat swab collecting process, the real-time information of all units communicates with the remote monitoring unit, and medical staff monitors the collected person and all the units in real time through a computer in a remote safety room, so that measures can be taken timely to guarantee the safety and effectiveness of the whole process when an accident occurs. The system realizes the full-automatic throat swab collection process.

Automatic throat swab sampling system

A takeoff and landing control method of a multimodal air-ground amphibious vehicle includes: receiving dynamic parameters of the multimodal air-ground amphibious vehicle; processing the dynamic parameters by a coupled dynamic model of the multimodal air-ground amphibious vehicle to obtain dynamic control parameters of the multimodal air-ground amphibious vehicle, wherein the coupled dynamic model of the multimodal air-ground amphibious vehicle comprises a motion equation of the multimodal air-ground amphibious vehicle in a touchdown state; and the motion equation of the multimodal air-ground amphibious vehicle in a touchdown state is determined by a two-degree-of-freedom suspension dynamic equation and a six-degree-of-freedom motion equation of the multimodal air-ground amphibious vehicle in the touchdown state; and controlling takeoff and landing of the multimodal air-ground amphibious vehicle according to the dynamic control parameters of the multimodal air-ground amphibious vehicle. The method is used for takeoff and landing control of a multimodal air-ground amphibious vehicle.

Takeoff and landing control method and control apparatus of multimodal air-ground amphibious vehicle, and computer storage medium

A 3D object detection method based on multi-view feature fusion of 4D RaDAR and LiDAR point clouds includes simultaneously acquiring RaDAR point cloud data and LiDAR point cloud data; and inputting the RaDAR point cloud data and the lidar point cloud data into a pre-established and trained RaDAR and LiDAR fusion network and outputting a 3D object detection result, wherein the RaDAR and LiDAR fusion network is configured to learn interaction information of a LiDAR and a RaDAR from a bird's eye view and a perspective view, respectively, and concatenate the interaction information to achieve fusion of the RaDAR point cloud data and the lidar point cloud data. The method can combine advantages of RaDAR and LiDAR, while avoiding disadvantages of the two modalities as much as possible to obtain a better 3D object detection result.

3D object detection method based on multi-view feature fusion of 4D RaDAR and LiDAR point clouds

An all-weather target detection method based on a vision and millimeter wave fusion includes: simultaneously acquiring continuous image data and point cloud data using two types of sensors of a vehicle-mounted camera and a millimeter wave radar; pre-processing the image data and point cloud data; fusing the pre-processed image data and point cloud data by using a pre-established fusion model, and outputting a fused feature map; and inputting the fused feature map into a YOLOv5 detection network for detection, and outputting a target detection result by non-maximum suppression. The method fully fuses millimeter wave radar echo intensity and distance information with the vehicle-mounted camera images. It analyzes different features of a millimeter wave radar point cloud and fuses the features with image information by using different feature extraction structures and ways, so that the advantages of the two types of sensor data complement each other.

All-weather target detection method based on vision and millimeter wave fusion

An automatic calibration method and system for spatial positions of a laser radar and a camera sensor is provided. The method includes: adjusting a spatial position of the laser radar relative to the camera sensor to obtain a plurality of spatial position relationships of the laser radar and the camera sensor; for a spatial position relationship, calculating a gray value of each laser radar point conforming to the line features after projection as a score, and accumulating scores of all laser radar points as a total score; traversing all the spatial position relationships to obtain a plurality of total scores; and selecting a spatial position relationship of the laser radar and the camera sensor corresponding to a highest total score from the plurality of total scores to serve as a calibrated position relationship of the laser radar and the camera sensor.

Automatic calibration method and system for spatial position of laser radar and camera sensor

A deep multimodal cross-layer intersecting fusion method, a terminal device and a storage medium are provided. The method includes: acquiring an RGB image and point cloud data containing lane lines, and pre-processing the RGB image and point cloud data; and inputting the pre-processed RGB image and point cloud data into a pre-constructed and trained semantic segmentation model, and outputting an image segmentation result. The semantic segmentation model is configured to implement cross-layer intersecting fusion of the RGB image and point cloud data. In the new method, a feature of a current layer of a current modality is fused with features of all subsequent layers of another modality, such that not only can similar or proximate features be fused, but also dissimilar or non-proximate features can be fused, thereby achieving full and comprehensive fusion of features. All fusion connections are controlled by a learnable parameter.

Deep multimodal cross-layer intersecting fusion method, terminal device, and storage medium

A LiDAR point cloud reflection intensity complementation method includes: acquiring a grayscale image and an original point cloud of a same road surface using a calibrated vehicle-mounted webcam and LiDAR; extracting edge information of the grayscale image using a preset edge extraction strategy, to obtain an edge image of the grayscale image; preprocessing the original point cloud to obtain an original point cloud reflection intensity projection image and an interpolated complementary point cloud reflection intensity projection image; and inputting the grayscale image, the edge image of the grayscale image, the original point cloud reflection intensity projection image and the interpolated complementary point cloud reflection intensity projection image into a pre-trained point cloud reflection intensity complementation model, and outputting a complementary point cloud reflection intensity projection image. A LiDAR point cloud reflection intensity complementation system is further provided.

LiDAR point cloud reflection intensity complementation method and system

An intelligent vehicle positioning method based on feature point calibration is provided. The intelligent vehicle positioning method includes: determining whether an intelligent vehicle is located in a blind area or a non-blind area; when the intelligent vehicle is located in a GNSS non-blind area, combining GNSS signals, odometer data and inertial measurement unit data, acquiring a current pose of the intelligent vehicle through a Kalman filtering, scanning a surrounding environment of the intelligent vehicle by using a laser radar, extracting corner points and arc features and performing processing to obtain the corner points and circle centers as the feature points, calculating global coordinates and weights of the feature points and storing the global coordinates and the weights of the feature points in a current feature point list.

Intelligent vehicle positioning method based on feature point calibration

A lane detection method based on vision and lidar multi-level fusion includes: calibrating obtained point cloud data and an obtained video image; constructing a point cloud clustering model by fusing height information, reflection intensity information of the point cloud data, and RGB information of the video image, obtaining point clouds of a road based on the point cloud clustering model, and obtaining a lane surface as a first lane candidate region by performing least square fitting on the point clouds; obtaining four-channel road information by fusing the reflection intensity information of the point cloud data and the RGB information of the video image, inputting the four-channel road information into the semantic segmentation network 3D-LaneNet, and outputting an image of a second lane candidate region; and fusing the first lane candidate region and the second lane candidate region, and combining the two lane candidate regions into a final lane region.

Lane detection method and system based on vision and lidar multi-level fusion

The present disclosure provides an electronic device and an information processing method. The electronic device comprises: a processor which comprises N processing units to process data and perform data input and output; a data processing interface coupled to Q processing units among the N processing units, which compresses raw data received from the Q processing units to obtain compressed data; and a memory coupled to the data processing interface, which receives and stores the compressed data, where N&#x2267;1 and 1&#x2266;Q&#x2266;N.

Growing community of inventors

Beijing, China

Huaping Liu