Ford Global Technolgoies, LLC

Ford Motor Company Limited

Gintaras Vincent Puskorius

Lee Albert Feldkamp

Mostafa Parchami

Bruno Sielly Jales Costa

Bryan Roger Goodman

Scott Andrew Amman

Francois Charette

Enrique Corona

Dimitar Petrov Filev

Vidya Nariyambut Murali

Guy Hotson

John V James

Kenneth A Marko

Melinda Kaye Hunsaker

Yu-Ning Liu

Brigitte Frances Mora Richardson

Gaurav Kumar Singh

John Edward Huber

Ranjani Rangarajan

Juan Enrique Castorena Martinez

James Beardslee

Veera V M L Ganesh Babu Alla

Ravindranatha Kundoor

Ali Hassani

Jinesh J Jain

Punarjay Chakravarty

Leighton Ira Davis, Jr

Timothy Mark Feldkamp

Anthony Dwayne Cooprider

Pavithra Madhavan

Kunjan Singh

Yakov M Fradkin

Gerald Jesion

Stephen Giardinelli

An Ji

Vahid Taimouri

Jian Lin

Joshua R Wheeler

Linjun Zhang

Gaurav Pandey

Helen Elizabeth Kourous-Harrigan

Codrin Cionca

Siddharth Agarwal

Gautham Sholingar

James W Butler

Daniel Levine

Faizan Shaik

Michael Alan Blommer

Yuksel Gur

Alan Peter Norton

Kyu Jeong Han

Raoul Tawel

Allan Miramonti

David Mark Newton

Chandana Neerukonda

Pol Llado

James Calvey Carnes

Mark Edward Porter

Kyoung Min Lee

Leda Daehler

Andrew Wagenmaker

Dinesh Palanisamy

Nazeeh Aranki

Christine A Gierdzak

Juan Castorena Martinez

Paul J Joseph Nicastri

Michel Cordonnier

Martin Pipoly

Robbie J Aniol

William Richard Fravel

Shigeru Sadakane

Martin Lenardon

He Li

Barbara Nance

A system includes a processor for performing one or more autonomous driving or assisted driving tasks based on a neural network. The neural network includes a base portion for performing feature extraction simultaneously for a plurality of tasks on a single set of input data. The neural network includes a plurality of subtask portions for performing the plurality of tasks based on feature extraction output from the base portion. Each of the plurality of subtask portions comprise nodes or layers of a neutral network trained on different sets of training data, and the base portion comprises nodes or layers of a neural network trained using each of the different sets of training data constrained by elastic weight consolidation to limit the base portion from forgetting a previously learned task.

Shared processing with deep neural networks

A computing system can receive, in a vehicle, moving object information is determined by processing lidar sensor data acquired by a stationary lidar sensor. The moving object information can be determined using typicality and eccentricity data analysis (TEDA) on the lidar sensor data. The vehicle can be operated based on the moving object information.

Vehicle path planning

A system includes a computer including a processor and a memory storing instructions executable by the processor to identify a location and an orientation of a vehicle on a map. The instructions include instructions to determine a location of an infrastructure sensor on the map based on the location and the orientation of the vehicle, data from a vehicle sensor, and data from the infrastructure sensor.

Sensor localization from external source data

The present invention extends to methods, systems, and computer program products for classifying time series image data. Aspects of the invention include encoding motion information from video frames in an eccentricity map. An eccentricity map is essentially a static image that aggregates apparent motion of objects, surfaces, and edges, from a plurality of video frames. In general, eccentricity reflects how different a data point is from the past readings of the same set of variables. Neural networks can be trained to detect and classify actions in videos from eccentricity maps. Eccentricity maps can be provided to a neural network as input. Output from the neural network can indicate if detected motion in a video is or is not classified as an action, such as, for example, a hand gesture.

Classifying time series image data

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to determine an eccentricity map based on video image data and determine vehicle motion data by processing the eccentricity map and two red, green, blue (RGB) video images with a deep neural network trained to output vehicle motion data in global coordinates. The instructions can further include instructions to operate a vehicle based on the vehicle motion data.

Vehicle visual odometry

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to detect a moving object in video stream data based on determining an eccentricity map. The instructions can further include instructions to determine a magnitude and direction of motion of the moving object, transform the magnitude and direction to global coordinates and operate a vehicle based on the transformed magnitude and direction.

Vehicle eccentricity mapping

A system, comprising a computer that includes a processor and a memory, the memory storing instructions executable by the processor to input a red-green-blue (RGB) image and an eccentricity image to a neural network which outputs a located object based on combining the RGB image and the eccentricity image, wherein the eccentricity image is based on a per-pixel rolling average and a per-pixel rolling variance over a moving window of k video frames. The memory can further include instructions executable by the processor to receive the located object at a computing device included in one or more of a vehicle or a traffic information system.

Eccentricity image fusion

A system, including a processor and a memory, the memory including instructions to be executed by the processor to identify first object features from sensor data acquired by a stationary sensor at a first time step, determine second object features at a second time step. The instructions can include further instructions to determine one or more object clusters of first object features by determining distances measured in pixels between the first object features and corresponding second object features and comparing the distances to one or more mean distances and determine one or more object groups of inlier first object features in the one or more object clusters by determining a plurality of similarity transformations for a plurality of random samples of first object features and determining inlier first object features based on maximizing the number of first object features included in a similarity transformation.

Group object-tracking

A system comprises a computer including a processor and a memory, and the memory including instructions such that the processor is programmed to calculate, at a server, a distance between a fixed camera disposed within an environment and a vehicle traversing the environment based on at least two image frames captured by the fixed camera, wherein the at least two image frames depict the vehicle at different image coordinates; determine a position of the fixed camera based on the distance and vehicle distance measurements measured by a vehicle sensor; and generate a map representing the environment based on the position of the fixed camera and the vehicle distance measurements.

Bearing only SLAM with cameras as landmarks

A system and methods are described for calibrating one sensor with respect to another. A method includes: determining a depth-motion vector using a first sensor; determining an optical-motion vector using a second sensor; and calibrating the first sensor with respect to the second sensor by minimizing a cost function that evaluates a distance between the depth-motion and optical-motion vectors.

Sensor calibration

A computing system can determine moving objects in a sequence of images based on recursively calculating red-green-blue (RGB) eccentricity 249based on a video data stream. A vehicle can be operated based on the determined moving objects. The video data stream can be acquired by a color video sensor included in the vehicle or a traffic infrastructure system.

Eccentricity maps

A system, comprising a computer that includes a processor and a memory, the memory storing instructions executable by the processor to determine an object location prediction based on a video data stream, wherein the object location prediction is based on processing cropped TEDA data with a neural network. The processor can be further programmed to download the object location prediction to a vehicle based on a location of the vehicle.

Vehicle object tracking

A computing system can crop an image based on a width, height and location of a first vehicle in the image. The computing system can estimate a pose of the first vehicle based on inputting the cropped image and the width, height and location of the first vehicle into a deep neural network. The computing system can then operate a second vehicle based on the estimated pose. The computing system may train a model to identify a type and a location of a hazard according to the estimated pose, the hazard being such things as ice, mud, pothole, or other hazard. The model may be used by an autonomous vehicle to identify and avoid hazards or to provide drive assistance alerts.

Road surface characterization using pose observations of adjacent vehicles

Growing community of inventors

Novi, MI, United States of America

Gintaras Vincent Puskorius