Ford Global Technolgoies, LLC

Ford Motor Company Limited

Jinesh J Jain

Harpreetsingh Banvait

Ashley Elizabeth Micks

Sneha Kadetotad

Marcos Paul Gerardo Castro

Vidya Nariyambut Murali

Daniel Levine

Brielle Reiff

Dongran Liu

Bruno Sielly Jales Costa

Kyu Jeong Han

Gautham Sholingar

Shreyasha Paudel

Scott Vincent Myers

Maryam Moosaei

Gintaras Vincent Puskorius

Adil Nizam Siddiqui

Gaurav Pandey

Nikita Jaipuria

Mohamed Ahmad

Praveen Narayanan

Guy Hotson

Lisa Scaria

Parsa Mahmoudieh

Peter Gyumyeong Joh

Madeline Jane Goh

Casey Bryan Feldman

Jinhyoung Oh

Aysha Habbaba

Xianling Zhang

Sowndarya Sundar

Ganesh Kumar

Lindsay Chan

Clifton Thomas

Sravani Yajamanam Kidambi

Connie Zeng

Leda Daehler

Smita Vinayak Bailur

Surjya Sarathi Ray

Katherine Stevo

Meghana Laxmidhar Gaopande

Hanieh Deilamsalehy

Marcos Pau Gerardo Castro

Harpreestsingh Banvait

Feng Jin

Sandhya Bhaskar

A computer includes a processor and a memory storing instructions executable by the processor to receive a dataset of images; extract feature data from the images; optimize a number of clusters into which the images are classified based on the feature data; for each cluster, optimize a number of subclusters into which the images in that cluster are classified; determine a metric indicating a bias of the dataset toward at least one of the clusters or subclusters based on the number of clusters, the numbers of subclusters, distances between the respective clusters, and distances between the respective subclusters; and after determining the metric, train a machine-learning program using a training set constructed from the clusters and the subclusters.

De-biasing datasets for machine learning

Systems, methods, and devices for estimating a shape of an object based on sensor data and determining a presence of a fault or a failure in a perception system. A method of the disclosure includes receiving sensor data from a range sensor and calculating a current shape reconstruction of an object based on the sensor data. The method includes retrieving from memory a prior shape reconstruction of the object based on prior sensor data. The method includes calculating a quality score for the current shape reconstruction by balancing a function of resulted variances of the current shape reconstruction and a similarity between the current shape reconstruction and the prior shape reconstruction.

Simultaneous diagnosis and shape estimation from a perceptual system derived from range sensors

Image data are input to a machine learning program. The machine learning program is trained with a virtual boundary model based on a distance between a host vehicle and a target object and a loss function based on a real-world physical model. An identification of a threat object is output from the machine learning program. A subsystem of the host vehicle is actuated based on the identification of the threat object.

Enhanced target detection

A system comprising a computer including a processor and a memory, the memory including instructions such that the processor is programmed to: process vehicle sensor data with a deep neural network to generate a prediction indicative of one or more objects based on the data and determine an object uncertainty corresponding to the prediction and when the object uncertainty is greater than an uncertainty threshold, segment the vehicle sensor data into a foreground portion and a background portion. Classify the foreground portion as including an unseen object class when a foreground uncertainty is greater than a foreground uncertainty threshold; classify the background portion as including unseen background when a background uncertainty is greater than a background uncertainty threshold; and transmit the data and a data classification to a server.

Unseen environment classification

A system can include a computer including a processor and a memory, the memory storing instructions executable by the processor to receive point cloud data. The instructions further include instructions to generate a plurality of feature maps based on the point cloud data, each feature map of the plurality of feature maps corresponding to a parameter of the point cloud data. The instructions further include instructions to aggregate the plurality of feature maps into an aggregated feature map. The instructions further include instructions to generate, via a feedforward neural network, at least one of a segmentation output or a classification output based on the aggregated feature map.

Segmentation and classification of point cloud data

A computer includes a processor and a memory storing instructions executable by the processor to collect a plurality of data sets, each data set from a respective sensor in a plurality of sensors, and each data set including a range, an azimuth angle, and a range rate for a detection point of the respective one of the sensors on an object to determine, for each detection point, a radial component of a ground speed of the detection point based on the data set associated with the detection point and a speed of a vehicle, and to generate a plurality of clusters, each cluster including selected detection points within a distance threshold from each other and having respective radial components of ground speeds that are (1) above a first threshold and (2) within a second threshold of each other.

Enhanced object detection with clustering

The present invention extends to methods, systems, and computer program products for formulating lane level routing plans. In general, aspects of the invention are used in motorized vehicles to guide a driver to a terminal vehicle configuration according to a lane level routing plan that balances travel time with routing plan robustness. A lane level routing plan can be based on terminal guidance conditions (e.g., exiting a highway in the correct off ramp lane), statistical patterns of lanes themselves, current vehicle state, and state of the local environment near the vehicle. Lane level routing plans can be communicated to the driver with audio, visual, and/or haptic cues. Lane level routing plans can be revised online and in (essentially) real-time in response to changing conditions in the local environment (e.g., a trailing vehicle in a neighboring lane has decided to increase speed).

Formulating lane level routing plans

A controller receives sensor data during a ride and provides it to a server system. A passenger further provides feedback concerning the ride in the form of some or all of an overall rating, flagging of ride anomalies, and flagging of road anomalies. The sensor data and feedback are input to a training algorithm, such as a deep reinforcement learning algorithm, which updates an artificial intelligence (AI) model. The updated model is then propagated to controllers of one or more autonomous vehicle which then perform autonomous navigation and collision avoidance using the updated AI model.

Feedback for an autonomous vehicle

An autonomous vehicle includes a microphone sensing sounds in the interior of the vehicle. The output of the interior microphone is processed according to an unsupervised machine learning model such that anomalies are indicated by the model. In response to detection of an anomaly, a remote dispatcher is notified, who may then dismiss the anomaly or transmit an instruction to the vehicle to alter its trajectory. The output of an exterior microphone and infotainment system may be removed from the output of the interior microphone prior to processing. An anomaly may be found to occur in response to detecting speaking of a keyword in the output of the interior microphone.

Detection of anomalies in the interior of an autonomous vehicle

A controller of an autonomous vehicle receives a travel itinerary of a passenger and an airport map. The controller then uses the airport map to arrive at a terminal corresponding to the passenger's travel itinerary. Upon arrival at the terminal the controller communicates with parked vehicles (V2V) or infrastructure to identify an unoccupied parking spot and then autonomously parks. When picking up a passenger, the controller determines whether the passenger has checked luggage and adjusts and arrival time accordingly and may account for the storage volume of the luggage. The controller may also loop a circuit at the airport where a wait time has been exceeded. In some embodiments, augmented reality may be used to help the passenger identify the vehicle.

Picking up and dropping off passengers at an airport using an autonomous vehicle

A controller receives outputs from a plurality of sensors such as a camera, LIDAR sensor, RADAR sensor, and ultrasound sensor, which may be rearward facing. A probability is updated each time a feature in a sensor output indicates presence of an object. The probability may be updated as a function of a variance of the sensor providing the output and a distance to the feature. Where the variance of a sensor is directional, directional probabilities may be updated according to these variances and the distance to the feature. If the probability meets a threshold condition, actions may be taken such as a perceptible alert or automatic braking. The probability may be decayed in the absence of detection of objects. Increasing or decreasing trends in the probability may be amplified by further increasing or decreasing the probability.

Multi-sensor probabilistic object detection and automated braking

A method is disclosed for using an autonomous vehicle to teach a student how to drive. The method may include generating, by the autonomous vehicle during a training session, a plan for navigating a section of road. During the training session, the autonomous vehicle may receive control instructions from the student regarding how the student would like to navigate the section of road. If the autonomous vehicle determines that implementing the instructions would be safe and legal, it may implement those instructions. However, if the autonomous vehicle determines that implementing the instructions would not be safe and legal, it may execute the plan. During a training session, an autonomous vehicle may provide on a head-up display to the student certain visual feedback regarding a driving performance of the student.

Autonomous vehicle providing driver education

A vehicle system includes a processor and a memory. The memory stores instructions executable by the processor to identify an area of interest from a plurality of areas on a map, to determine that a detected sound is received in a vehicle audio sensor upon determining that a source of the sound is within the area of interest and not another area in the plurality of areas, and to operate the vehicle based at least in part on the detected sound.

Object sound detection

A system comprises a computer that includes a processor and a memory. The memory stores instructions executable by the processor to estimate a first joint probability distribution of first data with respect to second data based on first and second marginal probability distributions of the first and second data, wherein the first data is from a first sensor based on a first sensor coordinate system, and the second data is from a second sensor based on a second sensor coordinate system, to estimate a second joint probability distribution of the first data with respect to the second data based on a projection of the first data onto a plane defined in the second sensor coordinate system, to estimate a rigid transformation between the first sensor coordinate system and the second sensor coordinate system by minimizing a distance between the first and second joint probability distributions, wherein the second joint probability distribution is a function of a set of extrinsic calibration parameters, and based on the set of the extrinsic calibration parameters, to detect an object in the first and second data.

Sensor fusion

Growing community of inventors

Palo Alto, CA, United States of America

Jinesh J Jain