Gm Global Technolgoy Operations, Inc.

Shaul Oron

Bat El Shlomo

Alon Raveh

Dan Levi

Ron M Hecht

Omer Tsimhoni

Tzach Ashkenazi

Noa Garnett

Max Bluvstein

Andrea Forgacs

Gershon Celniker

Liat Sless

Ethan Fetaya

Gilad Cohen

Ohad Rahamim

Roee Lahav

A method of training a disparity estimation network. The method includes obtaining an eye-gaze dataset having first images with at least one gaze direction associated with each of the first images. A gaze prediction neural network is trained based on the eye-gaze dataset to develop a model trained to provide a gaze prediction for an external image. A depth database is obtained that includes second images having depth information associated with each of the second images. A disparity estimation neural network for object detection is trained based on an output from the gaze prediction neural network and an output from the depth database.

System and method for eye-gaze direction-based pre-training of neural networks

A system and method for multi-camera vehicle top view object detection that includes capturing and sending video streams of a surrounding environment to a controller from multiple outward looking cameras that include each camera's detecting three-dimensional objects that are associated with that camera's coordinate frame, where one field of view partially overlaps another field of view. Each detected three-dimensional object from the outward looking cameras are transformed and projected onto a two-dimensional coordinate frame and image frame of each camera where a ranked two-dimensional affinity score between each of a pair of detected objects from the cameras are computed where if the affinity score for any object in each of the pair of detected objects is greater than a threshold amount, that object is removed from a list of ranked detected objects and an updated list of ranked detected objects is generated.

Multi-camera top view object detection and suppression

A vehicle, system and method of navigating the vehicle. The system includes a sensor and a processor. The sensor is configured to obtain a first set of detection points representative of a lane of a road section at a first time step and a second set of detection points representative of the lane at a second time step. The processor is configured to determine a set of predicted points for the second time step from the first set of detection points, obtain a set of fused points from the second set of detection points and the set of predicted points, and navigate the vehicle using the set of fused points.

Model free lane tracking system

Systems and methods include obtaining observation points of a lane line using a sensor of a vehicle. Each observation point indicates a location of a point on the lane line. A method includes generating or updating a lane model with the observation points. The lane model indicates a path of the lane line and the lane model is expressed in a lane-specific coordinate system that differs from a vehicle coordinate system that is defined by an orientation of the vehicle. The method also includes transforming the lane-specific coordinate system to maintain a correspondence between the lane-specific coordinate system and the vehicle coordinate system based on a change in orientation of the vehicle resulting in a change in the vehicle coordinate system.

Orientation-agnostic lane tracking in a vehicle

Systems and methods involve obtaining observation points of a lane line using one or more sensors of a vehicle. Each observation point indicates a location of a point on the lane line. A method includes obtaining uncertainty values, each uncertainty value corresponding with one of the observation points. A lane model is generated or updated using the observation points. The lane model indicates a path of the lane line. An uncertainty model is generated or updated using the uncertainty values corresponding with the observation points. The uncertainty model indicates uncertainty associated with each portion of the lane model.

Lane uncertainty modeling and tracking in a vehicle

A vehicle, system and method of detecting an object. The system includes an image network, a radar network and a head. The image network receives image data and proposes a boundary box from the image data and an object proposal. The radar network receives radar data and the boundary box and generates a fused set of data including the radar data and the image data. The head determines a parameter of the object from the object proposal and the fused set of data.

Object detection using low level camera radar fusion

Methods and apparatus are provided for detecting and assigning objects to sensed values. An object detection arrangement includes a processor that is programmed to execute a first branch of instructions and a second branch of instructions. Each branch of instructions includes receiving a modality from at least one sensor of a group of sensors via a respective interface and determining an output value based on the modality. The object detection arrangement includes an association distance matrix. Modalities of different branches of instructions define different modalities of an object external to the object detection arrangement. The object detection arrangement cumulates the output values, and the association distance matrix associates an object to the cumulated output values to thereby detect and track the object external to the object detection arrangement.

Learn association for multi-object tracking with multi sensory data and missing modalities

In one example implementation according to aspects of the present disclosure, a computer-implemented method includes projecting, by a processing device, tracked targets onto a virtual cylindrical omni-directional camera (VCOC). The method further includes projecting, by the processing device, detected targets onto the VCOC. The method further includes computing, by the processing device, a two-dimensional intersection-over-union (2D-IOU) between the tracked targets and the detected targets. The method further includes performing, by the processing device, an association between the tracked targets and the detected targets based at least in part on the computed IOU. The method further includes controlling, by the processing device, a vehicle based at least in part on the association.

Multi-sensor multi-object tracking

A method in a vehicle for performing multiple on-board sensing tasks concurrently in the same network using deep learning algorithms is provided. The method includes receiving vision sensor data from a sensor on the vehicle, determining a set of features from the vision sensor data using a plurality of feature layers in a convolutional neural network, and concurrently estimating, using the convolutional neural network, bounding boxes for detected objects, free-space boundaries, and object poses for detected objects from the set of features determined by the plurality of feature layers. The neural network may include a plurality of free-space estimation layers configured to determine the boundaries of free-space in the vision sensor data, a plurality of object detection layers configured to detect objects in the image and to estimate bounding boxes that surround the detected objects, and a plurality of object pose detection layers configured to estimate the direction of each object.

Growing community of inventors

Rehovot, Israel

Shaul Oron