Ford Global Technolgoies, LLC

Lsi Logic Corporation

Vsis, Inc.

Vijay K Nagasamy

Carlos Dangelo

Apurbaa Mallik

Gurjeet Singh

Aniruddh Ravindran

Hitha Revalla

Sreeranga P Rajan

Ahsan Bootehsaz

Zafar Iqbal

Michael D Rostoker

Ali Hassani

Vidya Nariyambut Murali

Praveen Narayanan

Nikita Jaipuria

Richard Deeley

Manoucher Vafai

James Sun

Mohammad Usman

Vijayanand Ponukumati

Dimitar Petrov Filev

Bruno Sielly Jales Costa

Gautham Sholingar

Doron Mintz

Rohan Bhasin

Kaushik Balakrishnan

Saeid Nooshabadi

Sowndarya Sundar

Steven Chao

Ali Mustafa

Deepti Mahajan

Rohun Atluri

J Carlos Dangelo

Manan Sanjay Patel

Yongbo Qian

Gaurab Banerjee

A computer includes a processor and a memory, and the memory stores instructions executable by the processor to receive an image frame from a camera, generate a feature map from the image frame, generate a depth map from the feature map, classify an object in the image frame based on the feature map, and estimate a distance to the object based on the depth map and based on an input to generating the feature map.

Distance determination from image data

Systems and methods for estimation of vehicle hitchball location are disclosed. A plurality of image frames may be received from a rear-facing camera of a vehicle. The rear-facing camera may be directed at a front of a trailer that is coupled to the vehicle at a hitchball. An approximate lateral location of the hitchball coupled to the vehicle may be determined by obtaining a plurality of cropped images along a hitch drawbar coupled to the vehicle by stepping along the vertical direction, and performing a stepwise lateral scan, centered at the hitch drawbar.

Systems and methods for a step and scan detection network for hitchball location estimation

A plurality of images can be acquired from a plurality of sensors and a plurality of flattened patches can be extracted from the plurality of images. An image location in the plurality of images and a sensor type token identifying a type of sensor used to acquire an image in the plurality of images from which the respective flattened patch was acquired can be added to each of the plurality of flattened patches. The flattened patches can be concatenated into a flat tensor and add a task token indicating a processing task to the flat tensor, wherein the flat tensor is a one-dimensional array that includes two or more types of data. The flat tensor can be input to a first deep neural network that includes a plurality of encoder layers and a plurality of decoder layers and outputs transformer output. The transformer output can be input to a second deep neural network that determines an object prediction indicated by the token and the object predictions can be output.

Sensor fusion

The disclosure is generally directed to systems and methods for inference quality determination of a deep neural network (DNN) without requiring ground truth information for use in driver-assisted vehicles, including receiving an image frame from a source; applying a normal inference DNN model to the image frame to produce a first inference with a first bounding box using a normal inference DNN model; applying a deep inference DNN model to a plurality of filtered versions of the image frame to produce a plurality of deep inferences with a plurality of bounding boxes; comparing the plurality of bounding boxes to identify a cluster condition of the plurality of bounding boxes; and determining an inference quality of the image frame of the normal inference DNN model as a function of the cluster condition.

Systems and methods for detecting deep neural network inference quality using image/data manipulation without ground truth information

A computer includes a processor and a memory storing instructions executable by the processor to receive image data from a camera, generate a depth map from the image data, detect an object in the image data, apply a bounding box circumscribing the object to the depth map, mask the depth map by setting depth values for pixels in the bounding box in the depth map to a depth value of a closest pixel in the bounding box, and determine a distance to the object based on the masked depth map. The closest pixel is closest to the camera of the pixels in the bounding box.

A first image can be acquired from a first sensor included in a vehicle and input to a deep neural network to determine a first bounding box for a first object. A second image can be acquired from the first sensor. Input latitudinal and longitudinal motion data from second sensors included in the vehicle corresponding to the time between inputting the first image and inputting the second image. A second bounding box can be determined by translating the first bounding box based on the latitudinal and longitudinal motion data. The second image can be cropped based on the second bounding box. The cropped second image can be input to the deep neural network to detect a second object. The first image, the first bounding box, the second image, and the second bounding box can be output.

Image annotation for deep neural networks

Embodiments describe a system configured with a brain machine interface (BMI) system implemented in a vehicle for performing vehicle functions using electrical impulses from motor cortex activity in a user's brain. The system uses fuzzy states for increased robustness. The fuzzy states are defined by sets of Gaussian kernel-type membership functions that are defined for steering and velocity action function states. The membership functions define fuzzy states that provide overlapping control tiers for increasing and decreasing vehicle functionality. An autonomous vehicle may perform control and governance of transitions between membership functions that may overlap, resulting in smooth transitioning between the states.

Analog driving feature control brain machine interface

Embodiments describe a vehicle configured with a brain machine interface (BMI) for a vehicle computing system to control vehicle functions using electrical impulses from motor cortex activity in a user's brain. A BMI training system trains the BMI device to interpret neural data generated by a motor cortex of a user and correlate the neural data to a vehicle control command associated with a neural gesture emulation function. A BMI system onboard the vehicle may receive a neural data feed of neural data from the user using the trained BMI device, determine, a user intention for a control instruction to control a vehicle infotainment system using the neural data feed, and perform an action based on the control instruction. The vehicle may further include a headrest configured as a Human Machine Interface (HMI) device that reads the electrical impulses without invasive electrode connectivity.

Vehicle computer command system with a brain machine interface

A system includes a computer including a processor and a memory, the memory storing instructions executable by the processor to generate a synthetic image by adjusting respective color values of one or more pixels of a reference image based on a specified meteorological optical range from a vehicle sensor to simulated fog, and input the synthetic image to a machine learning program to train the machine learning program to identify a meteorological optical range from the vehicle sensor to actual fog.

Enhanced vehicle operation

A training system for a deep neural network and method of training is disclosed. The system and/or method may comprise: receiving, from an eye-tracking system associated with a sensor, an image frame captured while an operator is controlling a vehicle; receiving, from the eye-tracking system, eyeball gaze data corresponding to the image frame; and iteratively training the deep neural network to determine an object of interest depicted within the image frame based on the eyeball gaze data. The deep neural network generates at least one feature map and determine a proposed region corresponding to the object of interest within the at least one feature map based on the eyeball gaze data.

Visual behavior guided object detection

A training system for a neural network system and method of training is disclosed. The method may comprise: receiving, from a sensor, an image frame captured while an operator is controlling a vehicle; using an eye-tracking system associated with the sensor, monitoring the eyes of the operator to determine eyeball gaze data; determining, from the image frame, a plurality of pedestrians; and iteratively training the neural network system to determine, from among the plurality of pedestrians, the one or more target pedestrians using the eyeball gaze data and an answer dataset that is based on the eyeball gaze data, wherein the determined one or more target pedestrians have a relatively-higher probability of collision with the vehicle than a remainder of the plurality of pedestrians.

Training a neural network to determine pedestrians

A system, including a processor and a memory, the memory including instructions to be executed by the processor to train a deep neural network based on a plurality of real-world images, determine the accuracy of the deep neural network is below a threshold based on identifying one or more physical features by the deep neural network, including one or more object types, in the plurality of real-world images and generate a plurality of synthetic images based on the accuracy of the deep neural network is below a threshold based on identifying the one or more physical features using a photo-realistic image rendering software program and a generative adversarial network. The instructions can include further instructions to retrain the deep neural network based on the plurality of real-world images and the plurality of synthetic images and output the retrained deep neural network.

Vehicle neural network

A trailer angle identification system comprises an imaging device configured to capture an image. An angle sensor is configured to measure a first angle of the trailer relative to a vehicle. A controller is configured to process the image in a neural network and estimate a second angle of the trailer relative to the vehicle based on the image. The controller is further configured to train the neural network based on a difference between the first angle and the second angle.

Trailer angle detection using end-to-end learning

A method for training an image-based trailer identification system comprises capturing a plurality of captured images in a field of view and identifying a detected trailer angle for a trailer in connection with a vehicle in each of the captured images. The method further comprises comparing the captured images and the corresponding trailer angles to a predetermined image set comprising a plurality of teaching trailer angles and identifying at least one required trailer angle of the teaching trailer angles that is not included in the captured images. Based on the captured images, a simulated angle image is generated. The simulated image comprises a depiction of the trailer in connection with the vehicle at the at least one required angle not included in the captured images. The method further comprises supplying the simulated angle image to the identification system for training.

A methodology for generating structural descriptions of complex digital devices from high-level descriptions and specifications is disclosed. The methodology uses a systematic technique to map and enforce consistency of the semantics imbedded in the intent of the original, high-level descriptions. The design activity is essentially a series of transformations operating upon various levels of design representations. At each level, the intended meaning (semantics) and formal software manipulations are captured to derive a more detailed level describing hardware meeting the design goals. Important features of the methodology are: capturing the users concepts, intent, specification, descriptions, constraints and trade-offs; architectural partitioning; what-if analysis at a high level; sizing estimation; timing estimation; architectural trade-off; conceptual design with implementation estimation; and timing closure. The methodology includes using estimators, based on data gathered over a number of realized designs, for partitioning and evaluating a design prior to logic synthesis. From the structural description, a physical implementation of the device is readily realized. A matrix of milestones (goals in the design activity) is defined by degree of complexity (level of abstraction) of a design and for progressive stages (levels) of design activity (from concept through implementation). The milestones are defined using continuous refinement, and the design activity proceeds towards subsequent milestones. As milestones are achieved, previous design activity becomes unalterable. A feasibility stage is key to convergence of the process. Single level or multi-level estimators determine the direction of the process.

Growing community of inventors

Mountain View, CA, United States of America

Vijay K Nagasamy