Probabilistic approach to unifying representations for robotic mapping

Abstract

The present disclosure provides methods, apparatuses, and computer-readable mediums for evaluating a reliability of three-dimensional shape predictions. In some embodiments, a method includes obtaining a scene representation including one or more images, estimating semantic information of partially-observed objects in the scene representation, based on geometric information extracted from the one or more images, determining segmented pointclouds of the partially-observed objects based on the semantic information and the geometric information, creating metric embeddings of the segmented pointclouds corresponding to the partially-observed objects, predicting completed shape pointclouds of the partially-observed objects, generating, using a probabilistic machine learning model, confidence scores for each point of the completed shape pointclouds, based on a correlation between the geometric information and the semantic information, and controlling a motion of a robot based on the confidence scores for each point of the completed shape pointclouds.