Microsoft Technology Licensing, LLC

Tsinghua University

Beijing University of Chemical Tehcnology

Beijing Xiaomi Mobile Software Co., Ltd.

Beijing Zitiao Network Technology Co., Ltd.

Beijing Bytedance Network Technology Co., Ltd.

Beihang University

Xiaomi Ev Technology Co., Ltd.

Zhiwei Li

Wei-Ying Ma

Mingjing Li

Rui Cai

Xinyu Zhang

Lei Zhang

Li Wang

Ming Li

Bin Wang

Jun Li

Wei Zhang

Huaping Liu

Shuming Shi

Qifan Tan

Kunfeng Wang

Tianyu Shen

Xing Xie

Ji-Rong Wen

Changhu Wang

Xingang Wu

Zhenhong Zou

Heung-Yeung Shum

Jue Wang

Hao Liu

Lie Lu

Arthur Charles Tomlin

Chi Zhang

Michael Scavezze

Kun Guo

Qing Yu

Xin Fan

Kuiyuan Yang

Dongfang Li

Yuanhao Chen

Hui Li

Hao Chen

Yang Zhou

Jing Liu

Lijun Zhao

Lei Zhu

Chao Wang

Yi Huang

Yong Rui

Hsiao-Wuen Hon

Lei Yang

Xintao Wu

Jiawei Gu

Zaiqing Nie

Hao Li

Li Tang

Tianlei Zhang

Lei Wu

Haohan Wu

Yunpeng Wang

Siyu He

Yuchao Liu

Qiang Hao

Xiaoguang Rui

Jingwei Wang

Weizheng Zhang

Haiyang Yu

Yiqian Lu

Zhiyu Zhang

Zhijun Xie

Xiaoming Xie

Zheng Fu

Huiqiao Zhang

Haiyu Qiao

Renbo Zhu

Lei Lai

Zhijian Zeng

Yongqing Jin

Jianxia You

Ruosen Hao

Weiye Xiao

Tingzhen Zhang

Fengli Lu

Zihao Wu

Peiran Chen

Jingshuo Liu

Wenbin Tang

Yilong Ren

Jing Lui

Jinyi Yao

Shiyan Zhang

Hanwen Gao

Rentao Sun

An incremental learning method and apparatus for a large Vision-Language Model for autonomous driving are provided. The incremental learning method includes: expanding a first training sample set to obtain a second training sample set, wherein the second training sample set includes a plurality of image samples annotated with road scene targets; inserting a plurality of fine-tuning sub-networks respectively into specified positions in a first large Vision-Language Model to generate a second large Vision-Language Model; processing the image samples in the second training sample set using the second large Vision-Language Model to obtain a target prediction result; calculating a loss value using the target prediction result and a target annotation result; and preserving parameters of the first large Vision-Language Model unchanged, and updating parameters of the fine-tuning sub-networks using the loss value.

Incremental learning method and apparatus for large vision-language model for autonomous driving

A method and an apparatus for collaborative end-to-end large model-oriented self-driving trajectory decision-making are provided. The method includes: processing an RGB image from ego vehicle, an RGB image from surrounding vehicle, and an RGB image from road-side by using a first feature extraction network to obtain a first image feature, a second image feature, and a third image feature, respectively; fusing first image feature, second image feature, and third image feature to obtain an image fusion feature; processing point cloud data of road-side to obtain a road-side point cloud feature; processing image fusion feature and road-side point cloud feature to obtain a first BEV feature and a second BEV feature; fusing first BEV feature and second BEV feature to obtain a fused BEV feature; and fusing prompt information and fused BEV feature to obtain text information, and then processing text information to obtain an ego vehicle trajectory decision-making result.

Method and apparatus for collaborative end-to-end large model-oriented self-driving trajectory decision-making

A multimodal perception decision-making method for autonomous driving based on a large language model includes: acquiring an RGB image and an infrared image of a target area at current time; processing the RGB image using a target detection model to obtain a predicted bounding box and a corresponding target detection category; processing the infrared image and the predicted bounding box and the corresponding target detection categories by using a segmentation model to obtain a target mask image; fusing the RGB image, the target mask image and the infrared image using a fusion model to obtain a fused feature map; performing fusion processing on first prompt information representing a user intent, second prompt information representing target detection category priorities, and the fused feature map, using a large Vision-Language Model to obtain textual information; and processing the textual information using a large natural language model to obtain a perception decision-making result.

Multimodal perception decision-making method and apparatus for autonomous driving based on large language model

Embodiments of the present disclosure relate to a video processing method and apparatus, a device, and a medium. The method includes: obtaining a reference editing operation sequence specified by a user, where the reference editing operation sequence includes at least one reference editing operation arranged in a sequential order of operations; and providing operation prompts according to the sequential order of the reference editing operations, to direct the user to perform the reference editing operations on a target video step by step, where each reference editing operation in the reference editing operation sequence has a corresponding operation prompt.

Video processing method and apparatus, device, and medium

A method and an apparatus for vehicle positioning. The method includes: acquiring a distance transformation image of a road marking object in a road image based on the road image of an environment where a vehicle is located at a current time point; acquiring a vector subgraph of the road marking object from a global vector map; and determining a target positioning posture of the vehicle based on the distance transformation image and the vector subgraph.

Method and apparatus for vehicle positioning

A method and apparatus for localizing an intelligent vehicle in a dynamic scene are provided. The localizing method includes: processing an RGB image using a target detection model to determine a rectangular box of a movable object; converting first point cloud data and second point cloud data to a pixel coordinate system, and dividing the movable object into a static object and a dynamic object; converting the second point cloud data of the movable object in the pixel coordinate system to a map coordinate system to obtain a semantic point cloud map; converting the second point cloud data of a communication area, and of a rectangular box, in the pixel coordinate system to the map coordinate system to obtain a static point cloud map; and determining observation weights for objects in the second point cloud data, and thereby determining pose information of the intelligent vehicle at the current time.

Method and apparatus for localizing intelligent vehicle in dynamic scene

Embodiments of the present disclosure relate to the technical field of computer processing, and provide an effect playback method and system for a live room, and a device. The method comprises: recognizing an article in a live room; determining a corresponding effect according to a target attribute of the article and the live room, wherein the correspondence among the article, the target attribute, and the effect is preset; and playing back the effect in the live room.

Method, system and device for playing effect in live room

A method for processing a map includes: obtaining a plurality of reference maps from vehicle-mounted devices; determining map elements included in the plurality of reference maps and a first element position where each map element is located; and obtaining a target map by fusing the map elements in the plurality of reference maps based on the first element position where each map element is located.

Methods for processing map, and vehicle

A vehicle localization method and device, an electronic device and a storage medium. The vehicle localization method includes: acquiring an image of a scene where a vehicle is located, the image comprising a semantic element; determining an element category corresponding to the semantic element; matching the semantic element with a map element to acquire a matching result by a matching method corresponding to the element category; and determining localization information of the vehicle according to the matching result.

Vehicle localization method and device, electronic device and storage medium

Display screen or portion thereof with a graphical user interface

A vision-LiDAR fusion method and system based on deep canonical correlation analysis are provided. The method comprises: collecting RGB images and point cloud data of a road surface synchronously; extracting features of the RGB images to obtain RGB features; performing coordinate system conversion and rasterization on the point cloud data in turn, and then extracting features to obtain point cloud features; inputting point cloud features and RGB features into a pre-established and well-trained fusion model at the same time, to output feature-enhanced fused point cloud features, wherein the fusion model fuses RGB features to point cloud features by using correlation analysis and in combination with a deep neural network; and inputting the fused point cloud features into a pre-established object detection network to achieve object detection. A similarity calculation matrix is utilized to fuse two different modal features.

Vision-LiDAR fusion method and system based on deep canonical correlation analysis

A target detection method based on the fusion of prior positioning of a millimeter-wave radar and a visual feature includes: simultaneously obtaining, based on the millimeter-wave radar and a vehicle-mounted camera after being calibrated, point cloud data of the millimeter-wave radar and a camera image; performing spatial 3D coordinate transformation on the point cloud data to project transformed point cloud data onto a camera plane; generating a plurality of anchor samples based on projected point cloud data according to a preset anchor strategy, and obtaining a final anchor sample based on a velocity-distance weight of each candidate region; fusing RGB information of the camera image and intensity information of an RCS in the point cloud data to obtain a feature of the final sample; and inputting the feature of the final anchor sample into a detection network to generate category and position information of a target in a scenario.

Target detection method based on fusion of prior positioning of millimeter-wave radar and visual feature

An all-weather target detection method based on a vision and millimeter wave fusion includes: simultaneously acquiring continuous image data and point cloud data using two types of sensors of a vehicle-mounted camera and a millimeter wave radar; pre-processing the image data and point cloud data; fusing the pre-processed image data and point cloud data by using a pre-established fusion model, and outputting a fused feature map; and inputting the fused feature map into a YOLOv5 detection network for detection, and outputting a target detection result by non-maximum suppression. The method fully fuses millimeter wave radar echo intensity and distance information with the vehicle-mounted camera images. It analyzes different features of a millimeter wave radar point cloud and fuses the features with image information by using different feature extraction structures and ways, so that the advantages of the two types of sensor data complement each other.

All-weather target detection method based on vision and millimeter wave fusion

A point cloud intensity completion method and system based on semantic segmentation are provided. The point cloud intensity completion method includes: acquiring an RGB image and point cloud data of a road surface synchronously by a photographic camera and a lidar; performing spatial transformation on the point cloud data by using a conversion matrix to generate a two-dimensional reflection intensity projection map and a two-dimensional depth projection map; performing reflection intensity completion on the RGB image and the two-dimensional reflection intensity projection map to obtain a single-channel reflection intensity projection map; performing depth completion on the RGB image and the two-dimensional depth projection map to obtain a single-channel depth projection map; and performing coarse-grained completion on the RGB image, the single-channel reflection intensity projection map and the single-channel depth projection map to obtain a two-dimensional coarse-grained reflectance intensity projection map.

Point cloud intensity completion method and system based on semantic segmentation

A lane line detection method using a fusion network based on an attention mechanism, and a terminal device are provided. The method includes: synchronously acquiring natural images and point cloud data of a road surface; and inputting the natural images and the point cloud data into a pre-built and trained fusion network to output a lane line detection result. Time series frames and an attention mechanism are added to the fusion network to perform information fusing on the point cloud data and the natural images. Specifically, continuous frames are used to improve detection network performance to deal with complex situations such as label loss and vehicle being blocked; low-dimensional features are stitched with high-dimensional features by Skip Connection to make up for image detail information that is continuously lost as the network goes deeper, and the Decoder module is used to restore image dimensions to obtain a final result.

Detection method using fusion network based on attention mechanism, and terminal device

Growing community of inventors

Beijing, China

Zhiwei Li