Tsinghua University

Chinese Academy of Sciences

Tsinghua Shenzhen International Graduate School

Microsoft Technology Licensing, LLC

Xiangyang Ji

Wen Gao

Yan Lu

Siwei Ma

Yongbing Zhang

Haoqian Wang

Debin Zhao

Shuncheng He

Kaifa Xin

Jizheng Xu

Feng Wu

Xin Jin

Lei Zhang

Wei Lin

Zhigang Li

Xizhi Huang

Yangguang Li

Yuhang Jiang

Jianzhun Shao

Guijin Wang

Gu Wang

Junhao Zheng

Honggang Qi

An encoded illumination real-time focusing scanning imaging device includes an LED array, an object stage, an objective lens, a liquid lens, a lens sleeve, and a camera. The object stage is used for placing a sample, the LED array is used for emitting bright field light or encoded light, and the bright field light or the encoded light respectively penetrates through the sample and then successively passes through the objective lens, the liquid lens, and the lens sleeve to reach the camera to shoot a bright field image or an encoded light illumination image by the camera. The present invention also discloses an encoded illumination real-time focusing scanning imaging method, which is performed by using the above-mentioned device. The present invention enables fast and accurate focusing at a low cost.

Encoded illumination real-time focusing scanning imaging device and method

Provided are a reinforcement learning-based label-free six-dimensional object pose prediction method and apparatus. The method includes: obtaining a target image to be predicted, the target image being a two-dimensional image including a target object; performing pose prediction based on the target image by using a pre-trained pose prediction model to obtain a prediction result, the pose prediction model being obtained by performing reinforcement learning based on a sample image; and determining a three-dimensional position and a three-dimensional direction of the target object based on the prediction result. The pose prediction model is trained by introducing reinforcement learning, the pose prediction is performed based on the target image by using the pre-trained pose prediction model, and thus the problem of six-dimensional object pose estimation based on two-dimensional images can be solved in the absence of real pose annotation, which ensures the prediction effect of label-free six-dimensional object pose prediction.

Reinforcement learning-based label-free six-dimensional object pose prediction method and apparatus

A high-resolution spectral image fast acquisition apparatus comprises an illumination source, an objective lens, a beam splitter, a single shot spectral image acquisition assembly and a reference image acquisition assembly, wherein the objective lens is used to align a sample to be measured; the illumination source is used to project an illumination light onto the sample to be measured so that the sample to be measured is amplified by the objective lens; wherein one part of amplified light enters the single shot spectral image acquisition assembly so as to acquire a low-resolution spectral cube of the sample to be measured, and another part of the amplified light enters the reference image acquisition assembly to acquire a high-resolution spectral cube. The apparatus enables rapid access to high-resolution spectral images, thereby speeding up the process of using spectral images for medical diagnosis.

High-resolution spectral image fast acquisition apparatus and method

The present disclosure discloses a multi-agent coordination method. The method includes: performing multiple data collections on N agents to collect E sets of data, where N and E are integers greater than 1; and optimizing neural networks of the N agents using reinforcement learning based on the E sets of data. Each data collection includes: randomly selecting a first coordination pattern from multiple predetermined coordination patterns; obtaining N observations after the N agents act on an environment in the first coordination pattern; determining a first probability and a second probability that a current coordination pattern is the first coordination pattern based on the N observations; and determining a pseudo reward based on the first probability and the second probability. The E sets of data include: a first coordination pattern label indicating the first coordination pattern, the N observations, and the pseudo reward.

Multi-agent coordination method and apparatus

The present disclosure discloses an unsupervised reinforcement learning method and apparatus based on Wasserstein distance. The method includes: obtaining a state distribution in a trajectory obtained with guidance of a current policy of an agent; calculating a Wasserstein distance between the state distribution and a state distribution in a trajectory obtained with another historical policy, and calculating a pseudo reward of the agent based on the Wasserstein distance, replacing a reward fed back from an environment in a target reinforcement learning framework with the pseudo reward, and guiding the current policy of the agent to keep a large distance from the other historical policy. The method uses Wasserstein distance to encourage an algorithm in an unsupervised reinforcement learning framework to obtain diverse policies and skills through training.

Unsupervised reinforcement learning method and apparatus based on Wasserstein distance

The present invention provides a ranging method based on laser-line scanning imaging to effectively suppress interference of extreme weather on imaging. The method includes the following steps: acquiring priori reference images for a fixed laser-line scanning system, including respectively placing reference whiteboards at different distances, projecting line laser beams to the whiteboards, and acquiring the reference images by using a camera; placing a laser-line scanning device in a real scene, causing the laser-line scanning device to respectively emit line lasers at different angles, and acquiring an image at each scanning angle by using a camera; and performing fusion calculation on the acquired scanning image in the real scene and the priori reference images by using a ranging algorithm based on laser-line scanning, and extracting distance information of a surrounding object, to implement environment perception.

Ranging method based on laser-line scanning imaging

Provided are a pose prediction method and apparatus, and a model training method and apparatus. The pose prediction method includes: performing target identification on a first image to be predicted to determine an area where a target object is located; determining a transformed target image based on the area where the target object is located; inputting the transformed target image into a pose decoupling prediction model for pose prediction; and determining a rotation amount of the target object based on an outputted result of a rotation amount branch network of the pose decoupling prediction model and determining a translation amount of the target object based on an outputted result of rotation amount branch network of the pose decoupling prediction model. By means of decoupling rotation and translation in object pose, the accuracy of pose prediction can be improved.

Pose prediction method and apparatus, and model training method and apparatus

The present disclosure relates to a method and an apparatus for training a 6D pose estimation network based on deep learning iterative matching. The method includes: obtaining a rendered image and a first segmentation mask of a target object by using a 3D model and an initial 6D pose estimation of the target object; inputting the rendered image, the first segmentation mask, an observed image of the target object, and a second segmentation mask of the target object in the observed image into a deep convolutional neural network to obtain a 6D pose estimation, a third segmentation mask and an optical flow; and performing said obtaining and said inputting again by updating the initial 6D pose estimation using the obtained relative 6D pose estimation and replacing the second segmentation mask with the third segmentation mask, to iteratively train the deep convolutional neural network.

Method and apparatus for training 6D pose estimation network based on deep learning iterative matching

The present invention discloses a depth map super-resolution processing method, including: firstly, respectively acquiring a first original image (S) and a second original image (S) and a low resolution depth map (d) of the first original image (S); secondly, 1) dividing the low resolution depth map (d) into multiple depth image blocks; 2) respectively performing the following processing on the depth image blocks obtained in step);) performing super-resolution processing on a current block with multiple super-resolution processing methods, to obtain multiple high resolution depth image blocks;) obtaining new synthesized image blocks by using an image synthesis technology;) upon matching and judgment, determining an ultimate high resolution depth image block; and 3) integrating the high resolution depth image blocks of the depth image blocks into one image according to positions of the depth image blocks in the low resolution depth map (d). Through the depth map super-resolution processing method of the present invention, depth information of the obtained high resolution depth maps is more accurate.

Depth map super-resolution processing method

Techniques and tools are described for scalable video coding and decoding. For example, a 3D sub-band video encoder includes an embedded base layer codec as well as temporal sub-band transforms and spatial sub-band transforms. The placement of the base layer codec among the sub-band transforms and the role of the base layer codec in scalable video coding vary depending on implementation. In general, the base layer codec provides efficient compression at low bit rates and produces a base layer compressed video bit stream compatible with existing decoders. At the same time, the 3D sub-band video encoder provides spatial and temporal scalability options at higher bit rates, refining the base layer video. A corresponding 3D sub-band video decoder includes an embedded base layer decoder.

Embedded base layer codec for 3D sub-band coding

An encoding method for skipped macroblocks in a video image includes the steps of: adding one indication bit into a picture header for indicating a coding mode for skipped macroblocks in a current image; selecting the coding mode for a macroblock type in the current image according to the number of skipped macroblocks, if it is a run_length coding, then setting the indication bit of the picture header as a status indicating a run_length coding, and encoding the macroblock type in the image by the run_length coding mode; if it is a joint coding, then setting the indication bit of the picture header as status indicating a joint coding and encoding the macroblock type in the image by the joint coding mode of the number of skipped macroblocks and the macroblock type; finally, encoding other data in the current macroblock and writing data into a code stream.

Encoding method for skipped macroblocks in video image

A method for obtaining an image reference block in a code mode of fixed reference frame number includes the steps of: performing motion estimation for each block of a current B frame and obtaining a motion vector MV of a corresponding block of a backward reference frame; discriminating whether the motion vector is beyond a maximum forward reference frame which is possibly pointed by the B frame, if not, then calculating the forward and backward motion vectors in a normal way; if yes, then using the motion vector of the forward reference frame that the B frame can obtain in the same direction to replace the motion vector of the corresponding block in the backward reference, and calculating the forward and the backward motion vectors of the B frame; finally, two image blocks pointed by the final obtained forward and backward motion vectors as the image reference blocks corresponding to the macro block. The present invention solves the possibly appeared problem of un-matching motion vectors, and can guarantee the coding efficiency to the largest extent.

Method for obtaining motion vector and image reference block in a code mode of fixed reference frame a number

The invention discloses a bi-directional prediction method for video coding/decoding. When bi-directional prediction coding at the coding end, firstly the given forward candidate motion vector of the current image block is obtained for every image block of the current B-frame; the backward candidate motion vector is obtained through calculation, and the candidate bi-directional prediction reference block is obtained through bi-directional prediction method; the match is computed within the given searching scope and/or the given matching threshold; finally the optimal matching block is selected to determine the final forward motion vector, and the backward motion vector and the block residual. The present invention achieves the object of bi-directional prediction by coding a single motion vector, furthermore, it will not enhance the complexity of searching for a matching block at the coding end, and may save amount of coding the motion vector and represent the motion of the objects in video more actually. The present invention realizes a new prediction coding type by combining the forward prediction coding with the backward.

Bi-directional predicting method for video coding/decoding

A 'rounding to zero' method can maintain the exact motion vector and can also be achieved by the method without division so as to improve the precision of calculating the motion vector, embody the motion of the object in video more factually, and obtain the more accurate motion vector prediction. Combining with the forward prediction coding and the backward prediction coding, the present invention realizes a new prediction coding mode, which can guarantee the high efficiency of coding in direct mode as well as is convenient for hardware realization, and gains the same effect as the conventional B frame coding.

Growing community of inventors

Beijing, China

Xiangyang Ji