Three-dimensional measurement method based on end-to-end deep learning for speckle projection

Abstract

The invention discloses a three-dimensional (3D) measurement method based on end-to-end deep learning for speckle projection. First, the speckle pattern was projected by the projector and collected simultaneously by the stereo camera. The speckle images after stereo rectification are fed into the stereo matching network. A feature extraction sub-network based on shared weights processes the speckle images to obtain a series of low-resolution 3D feature tensors. The feature tensor is fed into the saliency object detection sub-network to detect foreground information in the speckle images, producing a full-resolution valid mask map. A 4D matching cost volume is generated using the feature tensor of both views based on the candidate disparity range, filtered by a series of 3D convolutional layers to achieve cost aggregation, so that the initial disparity map is obtained by disparity regression. The final disparity map is obtained by combining the mask map and the initial disparity map to achieve a single-frame, robust, and absolute 3D shape measurement. The invention achieves a single-frame, robust, and absolute 3D shape measurement by projecting a single speckle pattern.