Ford Global Technolgoies, LLC

Gautham Sholingar

Vidya Nariyambut Murali

Nikita Jaipuria

Jinesh J Jain

Rohan Bhasin

Sowndarya Sundar

Gintaras Vincent Puskorius

Venkatapathi Raju Nallapa

Gaurav Pandey

Vijay K Nagasamy

Shreyasha Paudel

Akhil Perincherry

Sravani Yajamanam Kidambi

Leda Daehler

Deepti Mahajan

Anjali Krishnamachar

Shubh Gupta

A system, including a processor and a memory, the memory including instructions to be executed by the processor to receive one or more images from a vehicle, wherein a first deep neural network included in a computer in the vehicle has failed to determine an orientation of a first object in the one or more images. The instructions can include further instructions to generate a plurality of modified images with a few-shot image translator, wherein the modified images each include a modified object based on the first object. The instructions can include further instructions to re-train the deep neural network to determine the orientation of the first object based on the plurality of modified images and download the re-trained deep neural network to the vehicle.

Vehicle data augmentation

A system comprising a computer including a processor and a memory, the memory including instructions such that the processor is programmed to: process vehicle sensor data with a deep neural network to generate a prediction indicative of one or more objects based on the data and determine an object uncertainty corresponding to the prediction and when the object uncertainty is greater than an uncertainty threshold, segment the vehicle sensor data into a foreground portion and a background portion. Classify the foreground portion as including an unseen object class when a foreground uncertainty is greater than a foreground uncertainty threshold; classify the background portion as including unseen background when a background uncertainty is greater than a background uncertainty threshold; and transmit the data and a data classification to a server.

Unseen environment classification

A computer includes a processor and a memory, the memory storing instructions executable by the processor to apply a transform function to a plurality of images from a real-world dataset to generate a plurality of feature vectors, to apply a subspace generation algorithm to generate basis vectors of a subspace, and to project a simulated image onto the subspace to generate a realistic synthetic image.

Image enhancement for realism

A system, including a processor and a memory, the memory including instructions to be executed by the processor to train a deep neural network based on a plurality of real-world images, determine the accuracy of the deep neural network is below a threshold based on identifying one or more physical features by the deep neural network, including one or more object types, in the plurality of real-world images and generate a plurality of synthetic images based on the accuracy of the deep neural network is below a threshold based on identifying the one or more physical features using a photo-realistic image rendering software program and a generative adversarial network. The instructions can include further instructions to retrain the deep neural network based on the plurality of real-world images and the plurality of synthetic images and output the retrained deep neural network.

Vehicle neural network

The present disclosure discloses a system and a method. The system and the method generate, via a deep neural network, a first synthetic image based on a simulated image, generate a segmentation mask based on the synthetic image, compare the segmentation mask with a ground truth mask of the synthetic image, update the deep neural network based on the comparison, and generate, via the updated deep neural network, a second synthetic image based on the simulated image.

Training deep neural networks with synthetic images

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to generate a synthetic image and corresponding ground truth and generate a plurality of domain adapted synthetic images by processing the synthetic image with a variational auto encoder-generative adversarial network (VAE-GAN), wherein the VAE-GAN is trained to adapt the synthetic image from a first domain to a second domain. The instructions can include further instructions to train a deep neural network (DNN) based on the domain adapted synthetic images and the corresponding ground truth and process images with the trained deep neural network to determine objects.

Vehicle image generation

The present invention extends to methods, systems, and computer program products for evaluating autonomous vehicle algorithms. Aspects use (e.g., supervised) machine learning techniques to analyze performance of autonomous vehicle algorithms on real world and simulated data. Machine learning techniques can be used to identify scenario features that are more likely to influence algorithm performance. Machine learning techniques can also be used to consolidate insights and automate the generation of relevant test cases over multiple iterations to identify error-prone scenarios.

Evaluating autonomous vehicle algorithms

The present disclosure discloses a system and a method. In an example implementation, the system and the method can receive a synthetic image at a first deep neural network, and determine, via the first deep neural network, a prediction indicative of whether the synthetic image is machine-generated or is sourced from the real data distribution. The prediction can comprise a quantitative measure of photorealism of synthetic image.

Quantifying photorealism in simulated data with GANs

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to generate a pair of synthetic stereo images and a corresponding synthetic depth map with an image synthesis engine wherein the synthetic stereo images correspond to real stereo images acquired by a stereo camera and the synthetic depth map is a three-dimensional (3D) map corresponding to a 3D scene viewed by the stereo camera and process each image of the pair of synthetic stereo images pair independently using a generative adversarial network (GAN) to generate a fake image, wherein the fake image corresponds to one of the synthetic stereo images. The instructions can further include instructions to process a stereo pair of fake images to form a fake depth map, compare the fake depth map to the synthetic depth map using a Siamese one-shot neural network, train a deep neural network to determine an object using one or more of the fake images generated by the GAN, and operate a vehicle based on the object.

Vehicle image verification

A computing system can crop an image based on a width, height and location of a first vehicle in the image. The computing system can estimate a pose of the first vehicle based on inputting the cropped image and the width, height and location of the first vehicle into a deep neural network. The computing system can then operate a second vehicle based on the estimated pose. The computing system may train a model to identify a type and a location of a hazard according to the estimated pose, the hazard being such things as ice, mud, pothole, or other hazard. The model may be used by an autonomous vehicle to identify and avoid hazards or to provide drive assistance alerts.

Road surface characterization using pose observations of adjacent vehicles

The present invention extends to methods, systems, and computer program products for validating gesture recognition capabilities of automated systems. Aspects include a gesture recognition training system that is scalable, efficient, repeatable, and accounts for permutations of physical characteristics, clothing, types of gestures, environment, culture, weather, road conditions, etc. The gesture recognition training system includes sensors and algorithms used to generate training data sets that facilitate more accurate recognition of and reaction to human gestures. A training data set can be scaled from both monitoring and recording gestures performed by a humanoid robot and performed by animated humans in a simulation environment. From a scaled training data set, autonomous devices can be trained to recognize and react to a diverse set of human gestures in varying conditions with substantially improved capabilities. Recognition capabilities of an autonomous device can be validated and (re)trained until recognition capabilities are determined to be sufficient.

Validating gesture recognition capabilities of automated systems

A method for mitigating hazards to access to passenger vehicles. The method includes detecting, with one or more sensors, a hazardous condition in an area proximate a vehicle. A processor may calculate a safety metric corresponding to the hazardous condition and analyze the safety metric relative to a predetermined threshold. A vehicle occupant may be automatically notified of the hazardous condition in the event the safety metric satisfies the predetermined threshold. A corresponding system is also claimed herein.

Growing community of inventors

Sunnyvale, CA, United States of America

Gautham Sholingar