Ford Global Technolgoies, LLC

Nikita Jaipuria

Vidya Nariyambut Murali

Rohan Bhasin

Gautham Sholingar

Xianling Zhang

Punarjay Chakravarty

Praveen Narayanan

Gaurav Pandey

Jinesh J Jain

Vijay K Nagasamy

Shreyasha Paudel

Nathan Tseng

Meghana Laxmidhar Gaopande

Aniruddh Ravindran

Akhil Perincherry

Ramchandra Ganesh Karandikar

Hitha Revalla

Ganesh Kumar

Deepti Mahajan

Katherine Stevo

Artem Litvak

Sandhya Bhaskar

Eric Frankel

Mayar Arafa

Shubh Gupta

A computer includes a processor and a memory storing instructions executable by the processor to receive a dataset of images; extract feature data from the images; optimize a number of clusters into which the images are classified based on the feature data; for each cluster, optimize a number of subclusters into which the images in that cluster are classified; determine a metric indicating a bias of the dataset toward at least one of the clusters or subclusters based on the number of clusters, the numbers of subclusters, distances between the respective clusters, and distances between the respective subclusters; and after determining the metric, train a machine-learning program using a training set constructed from the clusters and the subclusters.

De-biasing datasets for machine learning

A computer that includes a processor and a memory, the memory including instructions executable by the processor to execute a multi-task learning (MTL) neural network that includes multi-task layers and single-task layers, and that is configured to receive an image as input to a first one of the multi-task layers and to output respective predictions for each of a plurality of tasks from respective ones of the single-task layers. Activation tensors can be output from the multi-task layers and can be provided as input to the single-task layers.

Multi-task learning

A trailer angle identification system includes an imaging device configured to capture an image. A controller is configured to receive steering angle data corresponding to a steering angle of a vehicle, receive vehicle speed data corresponding to a vehicle speed, and estimate an angle of the trailer relative to the vehicle by processing the image, the steering angle data, and the vehicle speed data in at least one neural network.

Systems and methods for trailer angle detection

A dual variational autoencoder-generative adversarial network (VAE-GAN) is trained to transform a real video sequence and a simulated video sequence by inputting the real video data into a real video decoder and a real video encoder and inputting the simulated video data into a synthetic video encoder and a synthetic video decoder. Real loss functions and simulated loss functions are determined based on output from a real video discriminator and a simulated video discriminator, respectively. The real loss functions are backpropagated through the real video encoder and the real video decoder to train the real video encoder and the real video decoder based on the real loss functions. The synthetic loss functions are backpropagated through the synthetic video encoder and the synthetic video decoder to train the synthetic video encoder and the synthetic video decoder based on the synthetic loss functions. The real video discriminator and the synthetic video discriminator can be trained to determine an authentic video sequence from a fake video sequence using the real loss functions and the synthetic loss functions. The annotated simulated video can be transformed with the synthetic video encoder and the real video decoder of the dual VAE-GAN to generate a reconstructed annotated real video sequence that includes style elements based on the real video sequence. A second neural network is trained using the reconstructed annotated real video sequence to detect and track objects.

Neural network for object detection and tracking

Image data are input to a machine learning program. The machine learning program is trained with a virtual boundary model based on a distance between a host vehicle and a target object and a loss function based on a real-world physical model. An identification of a threat object is output from the machine learning program. A subsystem of the host vehicle is actuated based on the identification of the threat object.

Enhanced target detection

A computer includes a processor and a memory storing instructions executable by the processor to receive a plurality of first images of an environment in a first lighting condition, classify pixels of the first images into categories, mask the pixels belonging to at least one of the categories from the first images, generate a three-dimensional representation of the environment based on the masked first images, and generate a second image of the environment in a second lighting condition based on the three-dimensional representation and on a first one of the first images.

Image relighting

A computer includes a processor and a memory storing instructions executable by the processor to receive a first image of a scene in a first lighting condition, generate a three-dimensional representation of the scene based on the first image, and generate a second image of the scene in a second lighting condition based on the three-dimensional representation and on the first image. The first image is an only image of the scene used for generating the three-dimensional representation. The first image is an only image of the scene used for generating the second image.

Single-perspective image relighting

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to input a fisheye image to a vector quantized variational autoencoder. The vector quantized variational autoencoder can encode the fisheye image to first latent variables based on an encoder. The vector quantized variational autoencoder can quantize the first latent variables to generate second latent variables based on a dictionary of embeddings. The vector quantized variational autoencoder can decode the second latent variables to a rectified rectilinear image using a decoder and output the rectified rectilinear image.

Image rectification

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to determine a second convolutional neural network (CNN) training dataset by determining an underrepresented object configuration and an underrepresented noise factor corresponding to an object in a first CNN training dataset, generate one or more simulated images including the object corresponding to the underrepresented object configuration in the first CNN training dataset by inputting ground truth data corresponding to the object into a photorealistic rendering engine and generate one or more synthetic images including the object corresponding to the underrepresented noise factor in the first CNN training dataset by processing the simulated images with a generative adversarial network (GAN) to determine a second CNN training dataset. The instructions can include further instructions to train a CNN to using the first and the second CNN training datasets.

Neural network image processing

The present disclosure discloses a system and a method that includes receiving, at a decoder, a latent representation of an image having a first domain, and generating a reconstructed image having a second domain, wherein the reconstructed image is generated based on the latent representation.

Domain translation network for performing image translation

The present disclosure discloses a system and a method. In an example implementation, the system and the method generate, at a first encoder neural network, an encoded representation of image features of an image received from a vehicle sensor of a vehicle. The system and method can also generate, at a second encoder neural network, an encoded representation of map tile features and generate, at the decoder neural network, a semantically segmented map tile based on the encoded representation of image features, the encoded representation of map tile features, and Global Positioning System (GPS) coordinates of the vehicle. The semantically segmented map tile includes a location of the vehicle and detected objects depicted within the image with respect to the vehicle.

Neural network for localization and object detection

A training system for a neural network system and method of training is disclosed. The method may comprise: receiving, from a sensor, an image frame captured while an operator is controlling a vehicle; using an eye-tracking system associated with the sensor, monitoring the eyes of the operator to determine eyeball gaze data; determining, from the image frame, a plurality of pedestrians; and iteratively training the neural network system to determine, from among the plurality of pedestrians, the one or more target pedestrians using the eyeball gaze data and an answer dataset that is based on the eyeball gaze data, wherein the determined one or more target pedestrians have a relatively-higher probability of collision with the vehicle than a remainder of the plurality of pedestrians.

Training a neural network to determine pedestrians

A system, including a processor and a memory, the memory including instructions to be executed by the processor to train a deep neural network based on a plurality of real-world images, determine the accuracy of the deep neural network is below a threshold based on identifying one or more physical features by the deep neural network, including one or more object types, in the plurality of real-world images and generate a plurality of synthetic images based on the accuracy of the deep neural network is below a threshold based on identifying the one or more physical features using a photo-realistic image rendering software program and a generative adversarial network. The instructions can include further instructions to retrain the deep neural network based on the plurality of real-world images and the plurality of synthetic images and output the retrained deep neural network.

Vehicle neural network

A computer includes a processor and a memory, the memory storing instructions executable by the processor to input an image to a first layer of a machine learning program, the first layer trained to identify one or more quadrilateral regions in the image, upon identifying the one or more quadrilateral regions, input the collected image to a second layer of a machine learning program, the second layer trained to identify a plurality of sets of vertices, each set of vertices defining a respective polygonal area, identify one of the polygonal areas in which to park a vehicle, and actuate one or more vehicle components to move the vehicle into the identified polygonal area.

Enhanced vehicle operation

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to generate two or more stereo pairs of synthetic images and generate two or more stereo pairs of real images based on the two or more stereo pairs of synthetic images using a generative adversarial network (GAN), wherein the GAN is trained using a six-axis degree of freedom (DoF) pose determined based on the two or more pairs of real images. The instructions can further include instructions to train a deep neural network based on a sequence of real images and operate a vehicle using the deep neural network to process a sequence of video images acquired by a vehicle sensor.

Growing community of inventors

Union City, CA, United States of America

Nikita Jaipuria