Siemens Aktiengesellschaft

Toyota Research Institute, Inc.

Woven Planet North America, Inc.

Sergey Zakharov

Wadim Kehl

Slobodan Ilic

Adrien David Gaidon

Ziyan Wu

Benjamin Planche

Vitor Guizilini

Andreas Hutter

Shadi Albarqouni

Linda Mai Bui

Systems and methods for training an adapter network that adapts a model pre-trained on synthetic images to real-world data are disclosed herein. A system may include a processor and a memory in communication with the processor and having machine-readable that cause the processor to output, using a neural network, a predicted scene that includes a three-dimensional bounding box having pose information of an object, generate a rendered map of the object that includes a rendered shape of the object and a rendered surface normal of the object, and train the adapter network, which adapts the predicted scene to adjust for a deformation of the input image by comparing the rendered map to the output map acting as a ground truth.

System and method for training an adapter network to improve transferability to real-world datasets

A method for 3D auto-labeling of objects with predetermined structural and physical constraints includes identifying initial object-seeds for all frames from a given frame sequence of a scene. The method also includes refining each of the initial object-seeds over the 2D/3D data, while complying with the predetermined structural and physical constraints to auto-label 3D object vehicles within the scene. The method further includes linking the auto-label 3D object vehicles over time into trajectories while respecting the predetermined structural and physical constraints.

3D auto-labeling with structural and physical constraints

Systems and methods for three-dimensional object detection are disclosed herein. One embodiment inputs, to a neural network, a two-dimensional label associated with an object to produce a Normalized-Object-Coordinate-Space (NOCS) image and a shape vector, the shape vector mapping to a continuously traversable coordinate shape space (CSS); decodes the NOCS image and the shape vector to an object model in the CSS; back-projects, in a frustum, the NOCS image to a LIDAR point cloud; identifies correspondences between the LIDAR point cloud and the object model to estimate an affine transformation between the LIDAR point cloud and the object model; iteratively refines the affine transformation using a differentiable SDF renderer; extracts automatically a three-dimensional label for the object based, at least in part, on the iteratively refined affine transformation; and performs three-dimensional object detection of the object based, at least in part, on the extracted three-dimensional label for the object.

Systems and methods for 3D object detection

The disclosure relates to a method how to recover an object from a cluttered image. The disclosure also relates to a computer program product and a computer-readable storage medium including instructions which, when the program is executed by a computer, cause the computer to carry out the acts of the mentioned method. Further, the disclosure relates to methods how to train components of a recognition system for recovering an object from such a cluttered image. In addition, the disclosure relates to such a recognition system.

Object recognition from images using cad models as prior

A method of removing noise from a depth image includes presenting real-world depth images in real-time to a first generative adversarial neural network (GAN), the first GAN being trained by synthetic images generated from computer assisted design (CAD) information of at least one object to be recognized in the real-world depth image. The first GAN subtracts the background in the real-world depth image and segments the foreground in the real-world depth image to produce a cleaned real-world depth image. Using the cleaned image, an object of interest in the real-world depth image can be identified via the first GAN trained with synthetic images and the cleaned real-world depth image. In an embodiment the cleaned real-world depth image from the first GAN is provided to a second GAN that provides additional noise cancellation and recovery of features removed by the first GAN.

Segmenting and denoising depth images for recognition applications using generative adversarial neural networks

Various embodiments include a method for determining a pose of an object in its surroundings comprising: using an optical capture device to capture the object and its surroundings as current recording; determining the pose of the object using optical image analysis; and using a neural network to ascertain the pose of the object. The neural network is taught with multi-task learning using pose regression and descriptor learning using a triplet-wise loss function and a pair-wise loss function. The pose regression uses quaternions. Determining the triplet-wise loss function includes using a dynamic margin term. Determining the pair-wise loss function includes an anchoring function.

Growing community of inventors

Kirchseeon, Germany

Sergey Zakharov