Beijing University of Chemical Tehcnology

Tsinghua University

Qifan Tan

Wei Zhang

Li Wang

Zhiwei Li

Tianyu Shen

Kunfeng Wang

Jun Li

Xinyu Zhang

Huaping Liu

Hui Li

Yang Zhou

Xintao Wu

Haohan Wu

Xingang Wu

Jingwei Wang

Jianxi Luo

Weizheng Zhang

Zhiyu Zhang

Xiaoming Xie

Ruosen Hao

Weiye Xiao

Tingzhen Zhang

Fengli Lu

Zihao Wu

Peiran Chen

Jingshuo Liu

Kangyao Huang

An incremental learning method and apparatus for a large Vision-Language Model for autonomous driving are provided. The incremental learning method includes: expanding a first training sample set to obtain a second training sample set, wherein the second training sample set includes a plurality of image samples annotated with road scene targets; inserting a plurality of fine-tuning sub-networks respectively into specified positions in a first large Vision-Language Model to generate a second large Vision-Language Model; processing the image samples in the second training sample set using the second large Vision-Language Model to obtain a target prediction result; calculating a loss value using the target prediction result and a target annotation result; and preserving parameters of the first large Vision-Language Model unchanged, and updating parameters of the fine-tuning sub-networks using the loss value.

Incremental learning method and apparatus for large vision-language model for autonomous driving

A method and an apparatus for collaborative end-to-end large model-oriented self-driving trajectory decision-making are provided. The method includes: processing an RGB image from ego vehicle, an RGB image from surrounding vehicle, and an RGB image from road-side by using a first feature extraction network to obtain a first image feature, a second image feature, and a third image feature, respectively; fusing first image feature, second image feature, and third image feature to obtain an image fusion feature; processing point cloud data of road-side to obtain a road-side point cloud feature; processing image fusion feature and road-side point cloud feature to obtain a first BEV feature and a second BEV feature; fusing first BEV feature and second BEV feature to obtain a fused BEV feature; and fusing prompt information and fused BEV feature to obtain text information, and then processing text information to obtain an ego vehicle trajectory decision-making result.

Method and apparatus for collaborative end-to-end large model-oriented self-driving trajectory decision-making

A multimodal perception decision-making method for autonomous driving based on a large language model includes: acquiring an RGB image and an infrared image of a target area at current time; processing the RGB image using a target detection model to obtain a predicted bounding box and a corresponding target detection category; processing the infrared image and the predicted bounding box and the corresponding target detection categories by using a segmentation model to obtain a target mask image; fusing the RGB image, the target mask image and the infrared image using a fusion model to obtain a fused feature map; performing fusion processing on first prompt information representing a user intent, second prompt information representing target detection category priorities, and the fused feature map, using a large Vision-Language Model to obtain textual information; and processing the textual information using a large natural language model to obtain a perception decision-making result.

Multimodal perception decision-making method and apparatus for autonomous driving based on large language model

A method and apparatus for localizing an intelligent vehicle in a dynamic scene are provided. The localizing method includes: processing an RGB image using a target detection model to determine a rectangular box of a movable object; converting first point cloud data and second point cloud data to a pixel coordinate system, and dividing the movable object into a static object and a dynamic object; converting the second point cloud data of the movable object in the pixel coordinate system to a map coordinate system to obtain a semantic point cloud map; converting the second point cloud data of a communication area, and of a rectangular box, in the pixel coordinate system to the map coordinate system to obtain a static point cloud map; and determining observation weights for objects in the second point cloud data, and thereby determining pose information of the intelligent vehicle at the current time.

Method and apparatus for localizing intelligent vehicle in dynamic scene

A takeoff and landing control method of a multimodal air-ground amphibious vehicle includes: receiving dynamic parameters of the multimodal air-ground amphibious vehicle; processing the dynamic parameters by a coupled dynamic model of the multimodal air-ground amphibious vehicle to obtain dynamic control parameters of the multimodal air-ground amphibious vehicle, wherein the coupled dynamic model of the multimodal air-ground amphibious vehicle comprises a motion equation of the multimodal air-ground amphibious vehicle in a touchdown state; and the motion equation of the multimodal air-ground amphibious vehicle in a touchdown state is determined by a two-degree-of-freedom suspension dynamic equation and a six-degree-of-freedom motion equation of the multimodal air-ground amphibious vehicle in the touchdown state; and controlling takeoff and landing of the multimodal air-ground amphibious vehicle according to the dynamic control parameters of the multimodal air-ground amphibious vehicle. The method is used for takeoff and landing control of a multimodal air-ground amphibious vehicle.

Takeoff and landing control method and control apparatus of multimodal air-ground amphibious vehicle, and computer storage medium

A lane line detection method using a fusion network based on an attention mechanism, and a terminal device are provided. The method includes: synchronously acquiring natural images and point cloud data of a road surface; and inputting the natural images and the point cloud data into a pre-built and trained fusion network to output a lane line detection result. Time series frames and an attention mechanism are added to the fusion network to perform information fusing on the point cloud data and the natural images. Specifically, continuous frames are used to improve detection network performance to deal with complex situations such as label loss and vehicle being blocked; low-dimensional features are stitched with high-dimensional features by Skip Connection to make up for image detail information that is continuously lost as the network goes deeper, and the Decoder module is used to restore image dimensions to obtain a final result.

Detection method using fusion network based on attention mechanism, and terminal device

Growing community of inventors

Beijing, China

Qifan Tan