Qualcomm Incorporated

Radhika Dilip Gowaikar

Sundar Subramanian

Makesh Pravin John Wilson

Volodimir Slobodyanyuk

Amin Ansari

Ravi Teja Sukhavasi

Rashid Ahmed Akbar Attar

Christopher G Lott

Kiran Kumar Somasundaram

Onkar Jayant Dabeer

Donna Ghosh

Michael John Hamilton

Shantanu Chaisson Sanyal

Jing X Sun

Joseph Binamira Soriaga

Xinzhou Wu

Ahmed Kamel Sadek

Kambiz Azarian Yazdi

Christopher Brunner

Bence Major

Daniel Hendricus Franciscus Fontijne

Gerhard Reitmayr

Jayakrishnan Unnikrishnan

Vinod Viswanatha Menon

Michael Joshua Shomin

John Anthony Dougherty

Fu-Chun Yeh

Kambiz Azarian-Yazdi

Chunhao Chen

Behnaz Rezaei

Shweta Nandakumar

Teck Yian Lim

Christopher Gerard Lott

In some aspects, a device may receive sensor data associated with a vehicle and a set of frames. The device may aggregate, using a first pose, the sensor data associated with the set of frames to generate an aggregated frame, wherein the aggregated frame is associated with a set of cells. The device may obtain an indication of a respective occupancy label for each cell from the set of cells, wherein the respective occupancy label includes a first occupancy label or a second occupancy label, and wherein a subset of cells from the set of cells are associated with the first occupancy label. The device may train, using data associated with the aggregated frame, a machine learning model to generate an occupancy grid, based on a loss function that only calculates a loss for respective cells from the subset of cells. Numerous other aspects are described.

Machine learning based occupancy grid generation

A device for processing image data is disclosed. The device can obtain a radar point cloud and one or more frames of camera data. The device can determine depth estimates of one or more pixels of the one or more frames of camera data. The device can generate a pseudo lidar point cloud using the depth estimates of the one or more pixels of the one or more frames of camera data, wherein the pseudo lidar point cloud comprises a three-dimensional representation of at least one frame of the one or more frames of camera data. The device can determine one or more object bounding boxes based on the radar point cloud and the pseudo lidar point cloud.

Radar and camera data fusion

In some aspects, a system may receive, from a first one-dimensional radar array, first information based at least in part on first reflections associated with an azimuthal plane. The system may further receive, from a second one-dimensional radar array, second information based at least in part on second reflections associated with an elevation plane. Accordingly, the system may detect an object based at least in part on the first information and may determine an elevation associated with the object based at least in part on the second information. Numerous other aspects are described.

Object detection and ranging using one-dimensional radar arrays

The present disclosure generally relates to an object detection system. For example, aspects of the present disclosure relate to systems and techniques for performing object detection using sensor information, such as elevation and/or velocity information from one or more light-based sensors. One example apparatus generally includes one or more processors operably configured to: obtain sensor information indicating at least two objects in an environment; determine at least one of a velocity or an elevation associated with each object of the at least two objects; consolidate the at least two objects into a common object based on the at least one of the velocity or the elevation; and output an indication of the common object.

Sensor based object detection

According to some aspects of the disclosure, techniques for compression techniques for the radar data that can be used in real-time applications for automated or self-driving vehicles. One or more compression techniques can be selected and/or configured based on information regarding operational conditions provided by a central (vehicle) computer. Operational conditions can include environmental data (e.g., weather, traffic), processing capabilities, mode of operation, and more. Compression techniques can facilitate transport of compressed radar data from a radar sensor to the central computer for processing of the radar data for object detection, identification, positioning, etc.

Techniques for radar data compression

Disclosed are techniques for employing deep learning to analyze radar signals. In an aspect, an on-board computer of a host vehicle receives, from a radar sensor of the vehicle, a plurality of radar frames, executes a neural network on a subset of the plurality of radar frames, and detects one or more objects in the subset of the plurality of radar frames based on execution of the neural network on the subset of the plurality of radar frames. Further, techniques for transforming polar coordinates to Cartesian coordinates in a neural network are disclosed. In an aspect, a neural network receives a plurality of radar frames in polar coordinate space, a polar-to-Cartesian transformation layer of the neural network transforms the plurality of radar frames to Cartesian coordinate space, and the neural network outputs the plurality of radar frames in the Cartesian coordinate space.

Radar deep learning

Disclosed are techniques for fusing camera and radar frames to perform object detection in one or more spatial domains. In an aspect, an on-board computer of a host vehicle receives, from a camera sensor of the host vehicle, a plurality of camera frames, receives, from a radar sensor of the host vehicle, a plurality of radar frames, performs a camera feature extraction process on a first camera frame of the plurality of camera frames to generate a first camera feature map, performs a radar feature extraction process on a first radar frame of the plurality of radar frames to generate a first radar feature map, converts the first camera feature map and/or the first radar feature map to a common spatial domain, and concatenates the first radar feature map and the first camera feature map to generate a first concatenated feature map in the common spatial domain.

Early fusion of camera and radar frames

Embodiments provided herein allow for identification of one or more regions of interest in a radar return signal that would be suitable for selected application of super-resolution processing. One or more super-resolution processing techniques can be applied to the identified regions of interest. The selective application of super-resolution processing techniques can reduce processing requirements and overall system delay. The output data of the super-resolution processing can be provided to a mobile computer system. The output data of the super-resolution processing can also be used to reconfigure the radar radio frequency front end to beam form the radar signal in region of the detected objects. The mobile computer system can use the output data for implementation of deep learning techniques. The deep learning techniques enable the vehicle to identify and classify detected objects for use in automated driving processes. The super-resolution processing techniques can be implemented in analog and/or digital circuitry.

Super-resolution enhancement techniques for radar

A method performed by an apparatus is described. The method includes receiving map data that is based on first image data, second image data, and a similarity metric. The first image data can be received from a first vehicle and represent an object. The second image data can be received from a second vehicle and represent the object. The similarity metric can be associated with the object represented in the first image data and the object represented in the second image data. The method can also include storing, by a vehicle, the received map data and localizing the vehicle based on the stored map data.

Systems and methods for mapping based on multi-journey data

Disclosed are techniques for annotating image frames using information from a light detection and ranging (LiDAR) sensor. An exemplary method includes receiving, from the LiDAR sensor, at least one LiDAR frame, receiving, from a camera sensor, at least one image frame, removing LiDAR points that represent a ground surface of the environment, identifying LiDAR points of interest in the at least one LiDAR frame, segmenting the LiDAR points of interest to identify at least one object of interest in the at least one LiDAR frame, and annotating the at least one image frame with a three-dimensional oriented bounding box of the at least one object of interest detected in the at least one image frame by projecting the three-dimensional oriented bounding boxes from the at least one LiDAR frame to the at least one image frame using cross-calibration transforms between the LiDAR sensor and the camera.

Using light detection and ranging (LIDAR) to train camera and imaging radar deep learning networks

A method performed by an apparatus is described. The method includes receiving a first set of object data corresponding to a first journey. The method also includes receiving a second set of object data corresponding to a second journey. The method further includes determining a similarity metric between the first set of object data and the second set of object data. The similarity metric indicates a distance between the first set of object data and the second set of object data for at least one object. The method additionally includes clustering the first set of object data and the second set of object data for the at least one object based on the similarity metric to produce at least one object cluster. The method also includes producing map data based on the at least one object cluster.

A method performed by an apparatus is described. The method includes receiving a set of image frames including at least one object. The method also includes receiving a camera position for each image frame. The method further includes associating the at least one object between image frames based on one or more object points and the received camera position for each image frame to produce two-dimensional (2D) object location data. The method additionally includes estimating three-dimensional (3D) pose data of the at least one object based on the 2D object location data. The method also includes refining the 3D pose data based on a shape constraint.

Systems and methods for object location

Distributed antenna systems (DASs) can include a plurality of spatially separated remote antenna units. According to at least one example, a first group of remote antenna units can simulcast downlink transmissions on a first carrier with a particular sector identity (ID). A second group of remote antenna units, including at least one different remote antenna unit from the first group, can simulcast downlink transmissions on a second carrier with the same sector ID. According to at least one other example, two or more remote antenna units which include respective coverage areas that are non-adjacent to one another can be employed to simulcast downlink transmissions.

Growing community of inventors

San Diego, CA, United States of America

Radhika Dilip Gowaikar