Dalian University of Technology

Xin Yang

Xiaopeng Wei

Qiang Zhang

Baocai Yin

Haiyang Mei

Zhenjun Du

Yu Qiao

Wen Dong

Yang Wang

Tong Li

Felix Heide

Haiyin Piao

Xuefeng Yin

Bo Dong

Yuhao Liu

Dengping Fan

Yuanyuan Liu

Boyan Wei

Zhaoxuan Zhang

Jianchuan Ding

The present invention provides a robot dynamic obstacle avoidance method based on a multimodal spiking neural network. The present invention realizes a robot obstacle avoidance method in a dynamic environment by fusing laser radar data and processed event camera data and combining with the intrinsic learnable threshold of the spiking neural network for a scenario comprising dynamic obstacles. It solves the difficulty of failure of obstacle avoidance due to the difficulty in perceiving the dynamic obstacles in the obstacle avoidance task of a robot. The present invention helps the robot to fully perceive the static information and the dynamic information of the environment, uses the learnable threshold mechanism of the spiking neural network for efficient reinforcement learning training and decision making, and realizes autonomous navigation and obstacle avoidance in the dynamic environment. An event data enhanced model is combined to better adapt to the dynamic environment for obstacle avoidance.

Robot dynamic obstacle avoidance method based on multimodal spiking neural network

The present invention proposes an active scene mapping method based on constraint guidance and space optimization strategies, comprising a global planning stage and a local planning stage; in the global planning stage, the next exploration goal of a robot is calculated to guide the robot to explore a scene; and after the next exploration goal is determined, specific actions are generated according to the next exploration goal, the position of the robot and the constructed occupancy map in the local planning stage to drive the robot to go to a next exploration goal, and observation data is collected to update the information of the occupancy map. The present invention can effectively avoid long-distance round trips in the exploration process so that the robot can take account of information gain and movement loss in the exploration process, find a balance of exploration efficiency, and realize the improvement of active mapping efficiency.

Active scene mapping method based on constraint guidance and space optimization strategies

A method for obstacle avoidance in degraded environments of robots based on intrinsic plasticity of an SNN is disclosed. A decision network in a synaptic autonomous learning module takes lidar data, distance from a target point and velocity at a previous moment as state input, and outputs the velocity of left and right wheels of the robot through the autonomous adjustment of the dynamic energy-time threshold, so as to carry out autonomous perception and decision making. The method solves the difficulty of the lack of intrinsic plasticity in the SNN, which leads to the difficulty of adapting to degraded environments due to the homeostasis imbalance of the model, is successfully deployed in mobile robots to maintain a stable trigger rate for autonomous navigation and obstacle avoidance in degraded, disturbed and noisy environments, and has validity and applicability on different degraded scenes.

Method for obstacle avoidance in degraded environments of robots based on intrinsic plasticity of SNN

The present invention belongs to the technical field of 3D reconstruction in the field of computer vision, and provides a dataset generation method for self-supervised learning scene point cloud completion based on panoramas. Pairs of incomplete point cloud and target point cloud with RGB information and normal information can be generated by taking RGB panoramas, depth panoramas and normal panoramas in the same view as input for constructing a self-supervised learning dataset for training of the scene point cloud completion network. The key points of the present invention are occlusion prediction and equirectangular projection based on view conversion, and processing of the stripe problem and point-to-point occlusion problem during conversion. The method of the present invention includes simplification of the collection mode of the point cloud data in a real scene; occlusion prediction idea of view conversion; and design of view selection strategy.

Dataset generation method for self-supervised learning scene point cloud completion based on panoramas

A method for segmenting a camouflaged object image based on distraction mining is disclosed. PFNet successively includes a multi-layer feature extractor, a positioning module, and a focusing module. The multi-layer feature extractor uses a traditional feature extraction network to obtain different levels of contextual features; the positioning module first uses RGB feature information to initially determine the position of the camouflaged object in the image; the focusing module mines the information and removes the distraction information based on the image RGB feature information and preliminary position information, and finally determines the boundary of the camouflaged object step by step. The method of the present invention introduces the concept of distraction information into the problem of segmentation of the camouflaged object and develops a new information exploration and distraction information removal strategy to help the segmentation of the camouflaged object image.

Camouflaged object segmentation method with distraction mining

The present invention belongs to the technical field of computer vision, and provides a video semantic segmentation method based on active learning, comprising an image semantic segmentation module, a data selection module based on the active learning and a label propagation module. The image semantic segmentation module is responsible for segmenting image results and extracting high-level features required by the data selection module; the data selection module selects a data subset with rich information at an image level, and selects pixel blocks to be labeled at a pixel level; and the label propagation module realizes migration from image to video tasks and completes the segmentation result of a video quickly to obtain weakly-supervised data. The present invention can rapidly generate weakly-supervised data sets, reduce the cost of manufacture of the data and optimize the performance of a semantic segmentation network.

Video semantic segmentation method based on active learning

The invention belongs to scene segmentation's field in computer vision and is a depth-aware method for mirror segmentation. PDNet successively includes a multi-layer feature extractor, a positioning module, and a delineating module. The multi-layer feature extractor uses a traditional feature extraction network to obtain contextual features; the positioning module combines RGB feature information with depth feature information to initially determine the position of the mirror in the image; the delineating module is based on the image RGB feature information, combined with depth information to adjust and determine the boundary of the mirror. This method is the first method that uses both RGB image and depth image to achieve mirror segmentation in an image. The present invention has also been further tested. For mirrors with a large area in a complex environment, the PDNet segmentation results are still excellent, and the results at the boundary of the mirrors are also satisfactory.

Depth-aware method for mirror segmentation

The invention discloses a method for glass detection in a real scene, which belongs to the field of object detection. The present invention designs a combination method based on LCFI blocks to effectively integrate context features of different scales. Finally, multiple LCFI combination blocks are embedded into the glass detection network GDNet to obtain large-scale context features of different levels, thereby realize reliable and accurate glass detection in various scenarios. The glass detection network GDNet in the present invention can effectively predict the true area of glass in different scenes through this method of fusing context features of different scales, successfully detect glass with different sizes, and effectively handle with glass in different scenes. GDNet has strong adaptability to the various glass area sizes of the images in the glass detection dataset, and has the highest accuracy in the field of the same type of object detection.

Method for glass detection in real scenes

The invention belongs to the field of computer vision technology, and provides a fully automatic natural image matting method. For image matting of a single image, it is mainly composed of the extraction of high-level semantic features and low-level structural features, the filtering of pyramid features, the extraction of spatial structure information, and the late optimization of the discriminator network. The invention can generate accurate alpha matte without any auxiliary information, saving the time for scientific researchers to mark auxiliary information and the interaction time when users use it.

Growing community of inventors

Liaoning, China

Xin Yang