Microsoft Technology Licensing, LLC

Nvidia Corporation

Microsoft Corporation

Vaibhav Thukral

David Nister

Ibrahim Eden

Weerapan Wilairat

Bernard Kerr

Ruchi Bhargava

Morgan Kolya Venable

Chris Aholt

Jeffrey Liu

Vivek Pradeep

Djordje Nijemcevic

Michael Grabner

Amir Akbarzadeh

Mudit Agrawal

Matthew Ashman

Shivkumar Swaminathan

Bernard James Kerr

Timothy Andrew Large

Nigel Stuart Keam

Neda Cvijetic

Sudipta Narayan Sinha

Michael Kroepfl

Lev Cherkashin

Bill Chau

Christopher Maurice Mei

Rambod Jacoby

Ruchita Bhargava

Birgit Henke

Youding Zhu

Ivana Stojanovic

Yu Sheng

Jaeyoun Kim

Andrew Carley

Enliang Zheng

Russell Chreptyk

Vadim Cugunovs

Jordan Marr

Andy Campbell

Tharun Battula

Bhaven Dedhia

Akash Chandra Shekar

Huimin Guo

Nguyen Bach

David Zachris Nister

Embodiments of the present disclosure relate to encoding of junction area information in map data. In particular, the encoding may include organizing vehicle paths that traverse through a junction area according to path groups and organizing contentions that influence behavior of vehicles traveling along the vehicle paths according to contention groups. In addition, the encoding may include generating direction data structures that associate respective path groups with one or more of the contention groups. In these or other embodiments, the map data that corresponds to the junction area may be updated with direction data structures.

Encoding junction information in map data

In various examples, a method includes computing a current keyframe, the current keyframe being representative of an area around an autonomous vehicle at a current time based on map data. The method includes transforming a preceding keyframe to a coordinate frame of the autonomous vehicle at a first time prior to completing computation of the current keyframe to generate a first world model frame. The method includes transforming the preceding keyframe to the coordinate frame of the autonomous vehicle at a second time after the first time and prior to completing computation of the current keyframe to generate a second world model frame.

Keyframe-based compression for world model representation in autonomous systems and applications

Systems and methods for vehicle-based determination of HD map update information. Sensor-equipped vehicles may determine locations of various detected objects relative to the vehicles. Vehicles may also determine the location of reference objects relative to the vehicles, where the location of the reference objects in an absolute coordinate system is also known. The absolute coordinates of various detected objects may then be determined from the absolute position of the reference objects and the locations of other objects relative to the reference objects. Newly-determined absolute locations of detected objects may then be transmitted to HD map services for updating.

System and methods for updating high definition maps

An end-to-end system for data generation, map creation using the generated data, and localization to the created map is disclosed. Mapstreams&#x2014;or streams of sensor data, perception outputs from deep neural networks (DNNs), and/or relative trajectory data&#x2014;corresponding to any number of drives by any number of vehicles may be generated and uploaded to the cloud. The mapstreams may be used to generate map data&#x2014;and ultimately a fused high definition (HD) map&#x2014;that represents data generated over a plurality of drives. When localizing to the fused HD map, individual localization results may be generated based on comparisons of real-time data from a sensor modality to map data corresponding to the same sensor modality. This process may be repeated for any number of sensor modalities and the results may be fused together to determine a final fused localization result.

Map creation and localization for autonomous driving applications

Disclosed are an apparatus and a method of low-latency, low-power eye tracking. In some embodiments, the eye tracking method operates a first sensor having a first level of power consumption that tracks positions of an eye of a user. In response to detection that the eye does not change position for a time period, the method stops operation of the first sensor and instead operates a second sensor that detects a change of position of the eye. The second sensor has a level of power consumption lower than the level of power consumption of the first sensor. Once the eye position changes, the second sensor resumes operation.

Eye tracking using video information and electrooculography information

Systems and methods for controlling closed captioning using an eye tracking device are provided. The system for controlling closed captioning may comprise a display device, a closed captioning controller configured to display closed captioning text for a media item during playback on the display device, and an eye tracking device configured to detect a location of a user's gaze relative to the display device and send the location to the closed captioning controller. The closed captioning controller may be configured to recognize a predetermined gaze pattern of the user's gaze and, upon detecting the predetermined gaze pattern, partially or completely deemphasize the display of the closed captioning text.

Eye tracking enabled smart closed captioning

Examples relating to calibrating an estimated gaze location are disclosed. One example method comprises monitoring the estimated gaze location of a viewer using gaze tracking data from a gaze tracking system. Image data for display via a display device is received, the image data comprising at least one target visual and target visual metadata that identifies the at least one target visual. The target visual metadata is used to identify a target location of the at least one target visual. The estimated gaze location of the viewer is monitored and a probability that the viewer is gazing at the target location is estimated. The gaze tracking system is calibrated using the probability, the estimated gaze location and the target location to generate an updated estimated gaze location of the viewer.

Gaze detection calibration

Embodiments are disclosed herein that relate to selecting user interface elements via a periodically updated position signal. For example, one disclosed embodiment provides a method comprising displaying on a graphical user interface a representation of a user interface element and a representation of an interactive target. The method further comprises receiving an input of coordinates of the periodically updated position signal, and determining a selection of the user interface element if a motion interaction of the periodically updated position signal with the interactive target meets a predetermined motion condition.

Selecting user interface elements via position signal

A method to furnish input representing gaze direction in a computer system operatively coupled to a vision system. In this method, a first image of an eye at a first level of illumination is acquired by a camera of the vision system. The first image is obtained from the camera, and a second image of the eye corresponding to a second, different level of illumination is also obtained. Brightness of corresponding pixels of the first and second images is compared in order to distinguish a reflection of the illumination by the eye from a reflection of the illumination by eyewear. The input is then furnished based on the reflection of the illumination by the eye.

Gaze tracking through eyewear

Embodiments are disclosed for eye tracking systems and methods. An example eye tracking system comprises a plurality of light sources and a camera configured to capture an image of light from the light sources as reflected from an eye. The eye tracking system further comprises a logic device and a storage device storing instructions executable by the logic device to acquire frames of eye tracking data by iteratively projecting light from different combinations of light sources of the plurality of light sources and capturing an image of the eye during projection of each combination. The instructions may be further executable to select a selected combination of light sources for eye tracking based on a determination of occlusion detected in the image arising from a transparent or semi-transparent optical structure positioned between the eye and the camera and project light via the selected combination of light sources for eye tracking.

Handling glare in eye tracking

Examples relating calibrating an estimated gaze location are disclosed. One example method comprises monitoring the estimated gaze location of a viewer using gaze tracking data from a gaze tracking system. Image data for display via a display device is received and, without using input from the viewer, at least one target visual that may attract a gaze of the viewer and a target location of the target visual are identified within the image data. The estimated gaze location of the viewer is compared with the target location of the target visual. An offset vector is calculated based on the estimated gaze location and the target location. The gaze tracking system is calibrated using the offset vector.

Examples relating to using non-visual feedback to alert a viewer of a display that a visual change has been triggered are disclosed. One disclosed example provides a method comprising using gaze tracking data from a gaze tracking system to determine that a viewer changes a gaze location. Based on determining that the viewer changes the gaze location, a visual change is triggered and non-visual feedback indicating the triggering of the visual change is provided to the viewer. If a cancel change input is received within a predetermined timeframe, the visual change is not displayed. If a cancel change input is not received within the timeframe, the visual change is displayed via the display.

Non-visual feedback of visual change

Growing community of inventors

Kirkland, WA, United States of America

Vaibhav Thukral