Ford Global Technolgoies, LLC

Enrique Corona

Mostafa Parchami

Gintaras Vincent Puskorius

Kunjan Singh

Bruno Sielly Jales Costa

Gaurav Pandey

Faizan Shaik

Stephen Giardinelli

Dimitar Petrov Filev

Juan Enrique Castorena Martinez

Chandana Neerukonda

Ghassan AlRegib

Mohit Prabhushankar

Armin Parchami

Ryan Benkert

Xiaomin Li

A camera is positioned to obtain an image of an object. The image is input to a neural network that outputs a three-dimensional (3D) bounding box for the object relative to a pixel coordinate system and object parameters. Then a center of a bottom face of the 3D bounding box is determined in pixel coordinates. The bottom face of the 3D bounding box is located in a ground plane in the image. Based on calibration parameters for the camera that transform pixel coordinates into real-world coordinates, a) a distance from the center of the bottom face of the 3D bounding box to the camera relative to a real-world coordinate system and b) an angle between a line extending from the camera to the center of the bottom face of the 3D bounding box and an optical axis of the camera are determined. The calibration parameters include a camera height relative to the ground plane, a camera focal distance, and a camera tilt relative to the ground plane. A six degree-of-freedom (6DoF) pose for the object is determined based on the object parameters, the distance, and the angle.

Neural network object pose determination

A segmentation mask can be determined that includes at least one moving object in a plurality of images based on determining eccentricity for each pixel location in the plurality of images. A first image included in the plurality of images can be segmented by applying the segmentation mask to the image. The segmented first image can be transformed to a compressed dense matrix which includes pixel values for non-zero portions of the segmented first image. The compressed dense matrix can be input to a sparse convolutional neural network trained to detect objects. A detected object corresponding to the at least one moving object included in the first image can be output from the sparse convolutional neural network.

Object detection

A deep neural network (DNN) can be trained based on a first training dataset that includes first images including annotated first objects. The DNN can be tested based on the first training dataset to determine first object predictions including first uncertainties. The DNN can be tested by inputting a second training dataset and outputting first object predictions including second uncertainties, wherein the second training dataset includes second images including unannotated second objects. A subset of images included in the second training dataset can be selected based on the second uncertainties, The second objects in the selected subset of images included in the second training dataset can be annotated. The DNN can be trained based on the selected subset of images included in the second training dataset including the annotated second objects.

Neural network training

A computing system can receive, in a vehicle, moving object information is determined by processing lidar sensor data acquired by a stationary lidar sensor. The moving object information can be determined using typicality and eccentricity data analysis (TEDA) on the lidar sensor data. The vehicle can be operated based on the moving object information.

Vehicle path planning

A system, including a processor and a memory, the memory including instructions to be executed by the processor to determine camera calibration parameters including camera focal distance, camera height and camera tilt based on a two-dimensional calibration pattern located parallel to and coincident with a ground plane determined by a roadway surface and determine object data including object location, object speed and object direction for one or more objects in a field of view of the video camera.

Traffic camera calibration

A system, comprising a computer that includes a processor and a memory, the memory storing instructions executable by the processor to input a red-green-blue (RGB) image and an eccentricity image to a neural network which outputs a located object based on combining the RGB image and the eccentricity image, wherein the eccentricity image is based on a per-pixel rolling average and a per-pixel rolling variance over a moving window of k video frames. The memory can further include instructions executable by the processor to receive the located object at a computing device included in one or more of a vehicle or a traffic information system.

Eccentricity image fusion

A system includes a camera, a visual target including a GPS receiver, and a processor. The camera images the visual target at a plurality of locations to obtain a plurality of images. The processor receives GPS coordinates for each respective location. Also, the processor determines a 3D location of the visual target in a camera reference frame for each of the images. The processor derives a plurality of estimated GPS camera locations for each of the plurality of images and mitigates an error associated with each of the derived estimated GPS camera locations to derive an average GPS camera location. This location is used to determine and store a transformation usable to describe objects viewed by the camera in the camera reference frame in terms of the GPS reference frame.

Infrastructure node localization with respect to alternative frame of reference

A system, including a processor and a memory, the memory including instructions to be executed by the processor to identify first object features from sensor data acquired by a stationary sensor at a first time step, determine second object features at a second time step. The instructions can include further instructions to determine one or more object clusters of first object features by determining distances measured in pixels between the first object features and corresponding second object features and comparing the distances to one or more mean distances and determine one or more object groups of inlier first object features in the one or more object clusters by determining a plurality of similarity transformations for a plurality of random samples of first object features and determining inlier first object features based on maximizing the number of first object features included in a similarity transformation.

Group object-tracking

A computing system can receive an image including foreground pixels. The foreground pixels can be determined based on determining eccentricity &#x3b5;based on a sequence of images acquired by a stationary sensor. The vehicle can determine moving objects in the image based on the foreground pixels. The vehicle can be operated based on the moving objects in the image.

Foreground detection

A computing system can receive a stabilized image stream, wherein the stabilized image stream is drift-corrected based on determining that an input image stream is stable and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to the real world. The computing system can operate a vehicle based on determining at least one moving object in the stabilized image stream.

Growing community of inventors

Canton, MI, United States of America

Enrique Corona