Beijing University of Chemical Tehcnology

Tianyu Shen

Wei Zhang

Li Wang

Zhiwei Li

Qifan Tan

Kunfeng Wang

Hui Li

Yang Zhou

Xintao Wu

Huaping Liu

Haohan Wu

Weizheng Zhang

Jingwei Wang

Weiye Xiao

Tingzhen Zhang

Fengli Lu

Zihao Wu

Peiran Chen

Jingshuo Liu

Zhiyu Zhang

Xiaoming Xie

Ruosen Hao

An incremental learning method and apparatus for a large Vision-Language Model for autonomous driving are provided. The incremental learning method includes: expanding a first training sample set to obtain a second training sample set, wherein the second training sample set includes a plurality of image samples annotated with road scene targets; inserting a plurality of fine-tuning sub-networks respectively into specified positions in a first large Vision-Language Model to generate a second large Vision-Language Model; processing the image samples in the second training sample set using the second large Vision-Language Model to obtain a target prediction result; calculating a loss value using the target prediction result and a target annotation result; and preserving parameters of the first large Vision-Language Model unchanged, and updating parameters of the fine-tuning sub-networks using the loss value.

Incremental learning method and apparatus for large vision-language model for autonomous driving

A method and an apparatus for collaborative end-to-end large model-oriented self-driving trajectory decision-making are provided. The method includes: processing an RGB image from ego vehicle, an RGB image from surrounding vehicle, and an RGB image from road-side by using a first feature extraction network to obtain a first image feature, a second image feature, and a third image feature, respectively; fusing first image feature, second image feature, and third image feature to obtain an image fusion feature; processing point cloud data of road-side to obtain a road-side point cloud feature; processing image fusion feature and road-side point cloud feature to obtain a first BEV feature and a second BEV feature; fusing first BEV feature and second BEV feature to obtain a fused BEV feature; and fusing prompt information and fused BEV feature to obtain text information, and then processing text information to obtain an ego vehicle trajectory decision-making result.

Method and apparatus for collaborative end-to-end large model-oriented self-driving trajectory decision-making

A multimodal perception decision-making method for autonomous driving based on a large language model includes: acquiring an RGB image and an infrared image of a target area at current time; processing the RGB image using a target detection model to obtain a predicted bounding box and a corresponding target detection category; processing the infrared image and the predicted bounding box and the corresponding target detection categories by using a segmentation model to obtain a target mask image; fusing the RGB image, the target mask image and the infrared image using a fusion model to obtain a fused feature map; performing fusion processing on first prompt information representing a user intent, second prompt information representing target detection category priorities, and the fused feature map, using a large Vision-Language Model to obtain textual information; and processing the textual information using a large natural language model to obtain a perception decision-making result.

Multimodal perception decision-making method and apparatus for autonomous driving based on large language model

A method and apparatus for localizing an intelligent vehicle in a dynamic scene are provided. The localizing method includes: processing an RGB image using a target detection model to determine a rectangular box of a movable object; converting first point cloud data and second point cloud data to a pixel coordinate system, and dividing the movable object into a static object and a dynamic object; converting the second point cloud data of the movable object in the pixel coordinate system to a map coordinate system to obtain a semantic point cloud map; converting the second point cloud data of a communication area, and of a rectangular box, in the pixel coordinate system to the map coordinate system to obtain a static point cloud map; and determining observation weights for objects in the second point cloud data, and thereby determining pose information of the intelligent vehicle at the current time.

Method and apparatus for localizing intelligent vehicle in dynamic scene

Growing community of inventors

Beijing, China

Tianyu Shen