Nvidia Corporation

Microsoft Technology Licensing, LLC

Michael Grabner

David Nister

Ethan Eade

Frederik Schaffalitzky

Minwoo Park

Drew Edward Steedly

Michael Ebstyne

Ibrahim Eden

Vaibhav Thukral

Yue Wu

Ruchi Bhargava

Cheng-Chieh Yang

Jeremy Furtek

Jeffrey Liu

Christopher Douglas Edmonds

Daniel Joseph McCulloch

Arthur Charles Tomlin

Adam G Poulos

Neda Cvijetic

Jeffrey Alan Kohler

Anthony James Ambrus

Nicholas Fajt

Michael Kroepfl

Martin Thomas Shetter

David Fields

Jeroen Vanturennout

Michael Edward Samples

Lev Cherkashin

Constantin Dulu

Gavin Dean Lazarow

Matthew Sammis Ashman

Milan Vugdelija

Georg Klein

James Dack

Amir Akbarzadeh

Ivan Stojiljkovic

Eric Scott Rehmeyer

Birgit Henke

Tushar Cyril Bhatnagar

Youding Zhu

Yu Sheng

Ivana Stojanovic

Jonathan David Hildebrandt

Andrew Carley

Varun Ramesh Mani

Enliang Zheng

Alex Christopher Turner

Vadim Cugunovs

Edward Daniel Parker

Brent Charles Allen

Muhammad Jabir Kapasi

Miguel Angel Susffalich

Jordan Marr

Neena Kamath

Cecilia Bong

Michael John Ebstyne

In various examples, surface profile estimation and bump detection may be performed based on a three-dimensional (3D) point cloud. The 3D point cloud may be filtered in view of a portion of an environment including drivable free-space, and within a threshold height to factor out other objects or obstacles other than a driving surface and protuberances thereon. The 3D point cloud may be analyzed—e.g., using a sliding window of bounding shapes along a longitudinal or other heading direction—to determine one-dimensional (1D) signal profiles corresponding to heights along the driving surface. The profile itself may be used by a vehicle—e.g., an autonomous or semi-autonomous vehicle—to help in navigating the environment, and/or the profile may be used to detect bumps, humps, and/or other protuberances along the driving surface, in addition to a location, orientation, and geometry thereof.

Surface profile estimation and bump detection for autonomous machine

Systems and methods for performing visual odometry more rapidly. Pairs of representations from sensor data (such as images from one or more cameras) are selected, and features common to both representations of the pair are identified. Portions of bundle adjustment matrices that correspond to the pair are updated using the common features. These updates are maintained in register memory until all portions of the matrices that correspond to the pair are updated. By selecting only common features of one particular pair of representations, updated matrix values may be kept in registers. Accordingly, matrix updates for each common feature may be collectively saved with a single write of the registers to other memory. In this manner, fewer write operations are performed from register memory to other memory, thus reducing the time required to update bundle adjustment matrices and thus speeding the bundle adjustment process.

Visual odometry in autonomous machine applications

An end-to-end system for data generation, map creation using the generated data, and localization to the created map is disclosed. Mapstreams&#x2014;or streams of sensor data, perception outputs from deep neural networks (DNNs), and/or relative trajectory data&#x2014;corresponding to any number of drives by any number of vehicles may be generated and uploaded to the cloud. The mapstreams may be used to generate map data&#x2014;and ultimately a fused high definition (HD) map&#x2014;that represents data generated over a plurality of drives. When localizing to the fused HD map, individual localization results may be generated based on comparisons of real-time data from a sensor modality to map data corresponding to the same sensor modality. This process may be repeated for any number of sensor modalities and the results may be fused together to determine a final fused localization result.

Map creation and localization for autonomous driving applications

In various examples, surface profile estimation and bump detection may be performed based on a three-dimensional (3D) point cloud. The 3D point cloud may be filtered in view of a portion of an environment including drivable free-space, and within a threshold height to factor out other objects or obstacles other than a driving surface and protuberances thereon. The 3D point cloud may be analyzed&#x2014;e.g., using a sliding window of bounding shapes along a longitudinal or other heading direction&#x2014;to determine one-dimensional (1D) signal profiles corresponding to heights along the driving surface. The profile itself may be used by a vehicle&#x2014;e.g., an autonomous or semi-autonomous vehicle&#x2014;to help in navigating the environment, and/or the profile may be used to detect bumps, humps, and/or other protuberances along the driving surface, in addition to a location, orientation, and geometry thereof.

Surface profile estimation and bump detection for autonomous machine applications

In embodiments of a camera-based input device, the input device includes an inertial measurement unit that collects motion data associated with velocity and acceleration of the input device in an environment, such as in three-dimensional (3D) space. The input device also includes at least two visual light cameras that capture images of the environment. A positioning application is implemented to receive the motion data from the inertial measurement unit, and receive the images of the environment from the at least two visual light cameras. The positioning application can then determine positions of the input device based on the motion data and the images correlated with a map of the environment, and track a motion of the input device in the environment based on the determined positions of the input device.

Position-determining input device

A computing device is provided, including a display configured to display a plurality of holograms superimposed upon a physical environment. The computing device may further comprise a processor configured to store in non-volatile memory a representation of the physical environment, including a plurality of hologram anchors indicating locations at which the holograms are displayed. The processor may store a priority level of each hologram anchor, wherein each priority level is selected from a plurality of priority levels including a high priority level and a low priority level, and wherein at least one hologram anchor has the low priority level. The processor may determine that a total size of the plurality of hologram anchors exceeds a predetermined size threshold. The processor may, for at least one hologram anchor assigned the low priority level, delete that hologram anchor from the representation of the physical environment.

Hologram anchor prioritization

Embodiments related to mapping an environment of a machine-vision system are disclosed. For example, one disclosed method includes acquiring image data resolving one or more reference features of an environment and computing a parameter value based on the image data, wherein the parameter value is responsive to physical deformation of the machine-vision system.

3D mapping with flexible camera rig

Embodiments that relate to communicating to a user of a head-mounted display device an estimated quality level of a world-lock display mode are disclosed. For example, in one disclosed embodiment a sensor data is received from one or more sensors of the device. Using the sensor data, an estimated pose of the device is determined. Using the estimated pose, one or more virtual objects are displayed via the device in either the world-lock display mode or in a body-lock display mode. One or more of input uncertainty values of the sensor data and pose uncertainty values of the estimated pose are determined. The input uncertainty values and/or pose uncertainty values are mapped to the estimated quality level of the world-lock display mode. Feedback of the estimated quality level is communicated to a user via device.

World-locked display quality feedback

Architecture that enables optical character recognition (OCR) of text in video frames at the rate at which the frames are received. Additionally, conflation is performed on multiple text recognition results in the frame sequence. The architecture comprises an OCR text recognition engine and a tracker system; the tracker system establishes a common coordinate system in which OCR results from different frames may be compared and/or combined. From a set of sequential video frames, a keyframe is chosen from which the reference coordinate system is established. An estimated transformation from keyframe coordinates to subsequent video frames is computed using the tracker system. When text recognition is completed for any subsequent frame, the result coordinates can be related to the keyframe using the inverse transformation from the processed frame to the reference keyframe. The results can be rendered for viewing as the results are obtained.

Simultaneous tracking and text recognition in video frames

Growing community of inventors

Seattle, WA, United States of America

Michael Grabner