Nvidia Corporation

Sharp Laboratories of America, Inc

Qualcomm Incorporated

Hae-Jong Seo

Minwoo Park

David Nister

Neda Cvijetic

Junghyun Kwon

Berta Rodriguez Hervas

Xiaolin Lin

Abhishek Bajpayee

Sangmin Oh

Dongwoo Lee

Wenchao Zheng

Christopher A Segall

Petrus J L Van Beek

John Michael Wyrwas

Yilin Yang

Mehmet K Kocamaz

Davide Onofrio

Tae Eun Choe

Eric Jonathan Yuan

Xudong Chen

Paul Eric Jacobs

Evgeni Petrovich Gousev

Russell Wayne Gruhlke

Yu Wang

Alok Govil

Chang Yuan

Venkat Rangan

Xinyu Xu

Xiquan Cui

Nelson Rasquinha

Jacek Maitan

Igor Tryndin

Qiang Zhang

Chung-Po Huang

A neural network may be used to determine corner points of a skewed polygon (e.g., as displacement values to anchor box corner points) that accurately delineate a region in an image that defines a parking space. Further, the neural network may output confidence values predicting likelihoods that corner points of an anchor box correspond to an entrance to the parking spot. The confidence values may be used to select a subset of the corner points of the anchor box and/or skewed polygon in order to define the entrance to the parking spot. A minimum aggregate distance between corner points of a skewed polygon predicted using the CNN(s) and ground truth corner points of a parking spot may be used simplify a determination as to whether an anchor box should be used as a positive sample for training.

Object detection using skewed polygons suitable for parking space detection

In various examples, one or more deep neural networks (DNNs) are executed to regress on control points of a curve, and the control points may be used to perform a curve fitting operation—e.g., Bezier curve fitting—to identify landmark locations and geometries in an environment. The outputs of the DNN(s) may thus indicate the two-dimensional (2D) image-space and/or three-dimensional (3D) world-space control point locations, and post-processing techniques—such as clustering and temporal smoothing—may be executed to determine landmark locations and poses with precision and in real-time. As a result, reconstructed curves corresponding to the landmarks—e.g., lane line, road boundary line, crosswalk, pole, text, etc.—may be used by a vehicle to perform one or more operations for navigating an environment.

Landmark detection using curve fitting for autonomous driving applications

In various examples, a multi-sensor fusion machine learning model—such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

Sensor fusion for autonomous machine applications using machine learning

In various examples, systems and methods are disclosed that preserve rich spatial information from an input resolution of a machine learning model to regress on lines in an input image. The machine learning model may be trained to predict, in deployment, distances for each pixel of the input image at an input resolution to a line pixel determined to correspond to a line in the input image. The machine learning model may further be trained to predict angles and label classes of the line. An embedding algorithm may be used to train the machine learning model to predict clusters of line pixels that each correspond to a respective line in the input image. In deployment, the predictions of the machine learning model may be used as an aid for understanding the surrounding environment—e.g., for updating a world model—in a variety of autonomous machine applications.

Regression-based line detection for autonomous driving machines

In various examples, contrast values corresponding to pixels of one or more images generated using one or more sensors of a vehicle may be computed to detect and identify objects that trigger glare mitigating operations. Pixel luminance values are determined and used to compute a contrast value based on comparing the pixel luminance values to a reference luminance value that is based on a set of the pixels and the corresponding luminance values. A contrast threshold may be applied to the computed contrast values to identify glare in the image data to trigger glare mitigating operations so that the vehicle may modify the configuration of one or more illumination sources so as to reduce glare experienced by occupants and/or sensors of the vehicle.

Glare mitigation using image contrast analysis for autonomous systems and applications

In various examples, a path perception ensemble is used to produce a more accurate and reliable understanding of a driving surface and/or a path there through. For example, an analysis of a plurality of path perception inputs provides testability and reliability for accurate and redundant lane mapping and/or path planning in real-time or near real-time. By incorporating a plurality of separate path perception computations, a means of metricizing path perception correctness, quality, and reliability is provided by analyzing whether and how much the individual path perception signals agree or disagree. By implementing this approach—where individual path perception inputs fail in almost independent ways—a system failure is less statistically likely. In addition, with diversity and redundancy in path perception, comfortable lane keeping on high curvature roads, under severe road conditions, and/or at complex intersections.

Path perception diversity and redundancy in autonomous machine applications

In various examples, a multi-sensor fusion machine learning model&#x2014;such as a deep neural network (DNN)&#x2014;may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

In various examples, one or more deep neural networks (DNNs) are executed to regress on control points of a curve, and the control points may be used to perform a curve fitting operation&#x2014;e.g., Bezier curve fitting&#x2014;to identify landmark locations and geometries in an environment. The outputs of the DNN(s) may thus indicate the two-dimensional (2D) image-space and/or three-dimensional (3D) world-space control point locations, and post-processing techniques&#x2014;such as clustering and temporal smoothing&#x2014;may be executed to determine landmark locations and poses with precision and in real-time. As a result, reconstructed curves corresponding to the landmarks&#x2014;e.g., lane line, road boundary line, crosswalk, pole, text, etc.&#x2014;may be used by a vehicle to perform one or more operations for navigating an environment.

In various examples, systems and methods are disclosed that preserve rich spatial information from an input resolution of a machine learning model to regress on lines in an input image. The machine learning model may be trained to predict, in deployment, distances for each pixel of the input image at an input resolution to a line pixel determined to correspond to a line in the input image. The machine learning model may further be trained to predict angles and label classes of the line. An embedding algorithm may be used to train the machine learning model to predict clusters of line pixels that each correspond to a respective line in the input image. In deployment, the predictions of the machine learning model may be used as an aid for understanding the surrounding environment&#x2014;e.g., for updating a world model&#x2014;in a variety of autonomous machine applications.

In various examples, a path perception ensemble is used to produce a more accurate and reliable understanding of a driving surface and/or a path there through. For example, an analysis of a plurality of path perception inputs provides testability and reliability for accurate and redundant lane mapping and/or path planning in real-time or near real-time. By incorporating a plurality of separate path perception computations, a means of metricizing path perception correctness, quality, and reliability is provided by analyzing whether and how much the individual path perception signals agree or disagree. By implementing this approach&#x2014;where individual path perception inputs fail in almost independent ways&#x2014;a system failure is less statistically likely. In addition, with diversity and redundancy in path perception, comfortable lane keeping on high curvature roads, under severe road conditions, and/or at complex intersections, as well as autonomous negotiation of turns at intersections, may be enabled.

In various examples, a deep neural network (DNN) is trained for sensor blindness detection using a region and context-based approach. Using sensor data, the DNN may compute locations of blindness or compromised visibility regions as well as associated blindness classifications and/or blindness attributes associated therewith. In addition, the DNN may predict a usability of each instance of the sensor data for performing one or more operations&#x2014;such as operations associated with semi-autonomous or autonomous driving. The combination of the outputs of the DNN may be used to filter out instances of the sensor data&#x2014;or to filter out portions of instances of the sensor data determined to be compromised&#x2014;that may lead to inaccurate or ineffective results for the one or more operations of the system.

Deep neural network processing for sensor blindness detection in autonomous machine applications

Example methods, systems, computer-readable media, and apparatuses for on-screen optical fingerprint capture for user authentication are presented. These allow for the authentication of a fingerprint where the fingerprint image is propagated in a glass layer before being captured by a camera. In some examples, such propagation can result in multiple fingerprint images resulting from total internal reflection within the glass layer. Feature information can then be determined from the captured image of the fingerprint, which can include multiple fingerprint images. The amount of feature information can then be reduced. A histogram associated with the captured image based on the reduced number of features can be generated, and a user can be authenticated based on the histogram.

Growing community of inventors

San Jose, CA, United States of America

Hae-Jong Seo