Ford Global Technolgoies, LLC

Mostafa Parchami

Codrin Cionca

Juan Enrique Castorena Martinez

Gintaras Vincent Puskorius

Enrique Corona

Gaurav Pandey

Zhen Zhao

Mahrdad Damsaz

Krithika Swaminathan

Kunjan Singh

Linjun Zhang

John Anthony Lockwood

Helen Elizabeth Kourous-Harrigan

Faizan Shaik

Chandana Neerukonda

Stephen Giardinelli

Jeffrey Thomas Remillard

Bruno Sielly Jales Costa

Siddharth Agarwal

Hamed Babazadehrokni

Punarjay Chakravarty

Shubham Shrivastava

Ankit Girish Vora

Sangjin Lee

Ghassan AlRegib

Mohit Prabhushankar

Bhushan Ghadge

Matthew Baer Wilmes

Vahid Taimouri

Alexandra Carlson

Ramanarayan Vasudevan

Rahul Kumar Agrawal

Spencer Carmichael

Katherine Skinner

Dinesh Palanisamy

Guoyu Lu

Ryan Benkert

Xiaomin Li

A computer that includes a processor and a memory, the memory including instructions executable by the processor to generate a current keyframe point cloud based on pairs of stereo images while a stereo camera travels through a scene to determine a similar viewpoint query matrix and an opposing viewpoint query matrix based the current keyframe point cloud. A distance matrix and an opposing view distance matrix can be generated by comparing the similar viewpoint query matrix and the opposing viewpoint query matrix to reference matrices. A relative pose between a stereo camera and a reference can be determined to determine a location in the scene during travel of the stereo camera through the scene by performing sequence matching in the distance matrix and the opposing view distance matrix to determine a minimum sequence.

System for operating a movable device

A first plurality of center points of first two-dimensional bounding boxes corresponding to a vehicle occurring in a first plurality of images acquired by a first camera can be determined. A second plurality of center points of second two-dimensional bounding boxes corresponding to the vehicle occurring in a second plurality of images acquired by a second camera can also be determined. A plurality of non-linear equations based on the locations of the first and second pluralities of center points and first and second camera parameters corresponding to the first and second cameras can be determined. The plurality of non-linear equations can be solved simultaneously for the locations of the vehicle with respect to the first and second cameras and the six degree of freedom pose of the second camera with respect to the first camera.

Camera calibration

A camera is positioned to obtain an image of an object. The image is input to a neural network that outputs a three-dimensional (3D) bounding box for the object relative to a pixel coordinate system and object parameters. Then a center of a bottom face of the 3D bounding box is determined in pixel coordinates. The bottom face of the 3D bounding box is located in a ground plane in the image. Based on calibration parameters for the camera that transform pixel coordinates into real-world coordinates, a) a distance from the center of the bottom face of the 3D bounding box to the camera relative to a real-world coordinate system and b) an angle between a line extending from the camera to the center of the bottom face of the 3D bounding box and an optical axis of the camera are determined. The calibration parameters include a camera height relative to the ground plane, a camera focal distance, and a camera tilt relative to the ground plane. A six degree-of-freedom (6DoF) pose for the object is determined based on the object parameters, the distance, and the angle.

Neural network object pose determination

A segmentation mask can be determined that includes at least one moving object in a plurality of images based on determining eccentricity for each pixel location in the plurality of images. A first image included in the plurality of images can be segmented by applying the segmentation mask to the image. The segmented first image can be transformed to a compressed dense matrix which includes pixel values for non-zero portions of the segmented first image. The compressed dense matrix can be input to a sparse convolutional neural network trained to detect objects. A detected object corresponding to the at least one moving object included in the first image can be output from the sparse convolutional neural network.

Object detection

A deep neural network (DNN) can be trained based on a first training dataset that includes first images including annotated first objects. The DNN can be tested based on the first training dataset to determine first object predictions including first uncertainties. The DNN can be tested by inputting a second training dataset and outputting first object predictions including second uncertainties, wherein the second training dataset includes second images including unannotated second objects. A subset of images included in the second training dataset can be selected based on the second uncertainties, The second objects in the selected subset of images included in the second training dataset can be annotated. The DNN can be trained based on the selected subset of images included in the second training dataset including the annotated second objects.

Neural network training

A system includes a shaft extendable from a vehicle roof, a sensor mount mounted to the shaft, a plurality of sensors supported by the sensor mount, and a computer communicatively coupled with one or more of the sensors to receive data from the one or more of the sensors, the computer including a processor and a memory. The memory stores instructions executable by the processor to extend the shaft to position the sensor mount to a specified height above the roof.

Vehicle roof assembly

A computing system can receive, in a vehicle, moving object information is determined by processing lidar sensor data acquired by a stationary lidar sensor. The moving object information can be determined using typicality and eccentricity data analysis (TEDA) on the lidar sensor data. The vehicle can be operated based on the moving object information.

Vehicle path planning

A system includes a computer including a processor and a memory storing instructions executable by the processor to identify a location and an orientation of a vehicle on a map. The instructions include instructions to determine a location of an infrastructure sensor on the map based on the location and the orientation of the vehicle, data from a vehicle sensor, and data from the infrastructure sensor.

Sensor localization from external source data

A sensor system is disclosed. The sensor system may comprise a housing; an emitter, carried by the housing, that emits a beam comprising depth-data signals; a beam-distribution adjustment system; and a processor programmed to control the adjustment system by selectively changing an angular distribution of the depth-data signals emitted from the housing.

Beam distribution adjustment for a sensor

A set of first candidate topologies of first candidate roadside infrastructure nodes at respective mounting locations in a geographic area is randomly generated. For each of the first candidate topologies, first simulations, including detection of objects according to selected sensor parameters, installation parameters, and environment parameters for the candidate nodes at the respective mounting locations, are executed. First fitness scores are determined for each of the first candidate topologies by comparing results of the first simulations to ground truth data. Upon identifying one of the first fitness scores as exceeding a threshold, the candidate topology associated with the identified first fitness score is identified for deployment.

Roadside infrastructure deployment

A reference pose of an object in a coordinate system of a map of an area is determined. The reference pose is based on a three-dimensional (3D) reference model representing the object. A first pose of the object is determined as the object moves with respect to the coordinate system. The first pose is determined based on the reference pose and sensor data collected by the sensor at a first time. A second pose of the object is determined as the object continues to move with respect to the coordinate system. The second pose is determined based on the reference pose, the first pose, and sensor data collected by the sensor at a second time consecutive to the first time.

Localizing a moving object

A system comprises a computer including a processor, and a memory. The memory stores instructions such that the processor is programmed to determine two or more clusters of vehicle operating parameter values from each of a plurality of vehicles at a location within a time. Determining the two or more clusters includes clustering data from the plurality of vehicles based on proximity to two or more respective means. The processor is further programmed to determine a reportable condition when a mean for a cluster representing a greatest number of vehicles varies from a baseline by more than a threshold.

Detecting changed driving conditions

A system, comprising a computer that includes a processor and a memory, the memory storing instructions executable by the processor to input a red-green-blue (RGB) image and an eccentricity image to a neural network which outputs a located object based on combining the RGB image and the eccentricity image, wherein the eccentricity image is based on a per-pixel rolling average and a per-pixel rolling variance over a moving window of k video frames. The memory can further include instructions executable by the processor to receive the located object at a computing device included in one or more of a vehicle or a traffic information system.

Eccentricity image fusion

A system includes a camera, a visual target including a GPS receiver, and a processor. The camera images the visual target at a plurality of locations to obtain a plurality of images. The processor receives GPS coordinates for each respective location. Also, the processor determines a 3D location of the visual target in a camera reference frame for each of the images. The processor derives a plurality of estimated GPS camera locations for each of the plurality of images and mitigates an error associated with each of the derived estimated GPS camera locations to derive an average GPS camera location. This location is used to determine and store a transformation usable to describe objects viewed by the camera in the camera reference frame in terms of the GPS reference frame.

Infrastructure node localization with respect to alternative frame of reference

A system, including a processor and a memory, the memory including instructions to be executed by the processor to identify first object features from sensor data acquired by a stationary sensor at a first time step, determine second object features at a second time step. The instructions can include further instructions to determine one or more object clusters of first object features by determining distances measured in pixels between the first object features and corresponding second object features and comparing the distances to one or more mean distances and determine one or more object groups of inlier first object features in the one or more object clusters by determining a plurality of similarity transformations for a plurality of random samples of first object features and determining inlier first object features based on maximizing the number of first object features included in a similarity transformation.

Growing community of inventors

Dearborn, MI, United States of America

Mostafa Parchami