Machine-trained network for misalignment-insensitive depth perception

Abstract

Some embodiments of the invention provide a novel method for training a multi-layer node network to reliably determine depth based on a plurality of input sources (e.g., cameras, microphones, etc.) that may be arranged with deviations from an ideal alignment or placement. Some embodiments train the multi-layer network using a set of inputs generated with random misalignments incorporated into the training set. In some embodiments, the training set includes (i) a synthetically generated training set based on a three-dimensional ground truth model as it would be sensed by a sensor array from different positions and with different deviations from ideal alignment and placement, and/or (ii) a training set generated by a set of actual sensor arrays augmented with an additional sensor (e.g., additional camera or time of flight measurement device such as lidar) to collect ground truth data.