Plusai Limited

Magic Leap, Inc.

Yahoo! Inc.

Oath Inc.

Symbol Technologies, LLC

Excalibur Ip, LLC

Yahoo Assets LLC

Hao Zheng

David Wanqian Liu

Timothy Patrick Daly, Jr

Zhiheng Jia

Anurag Ganguli

Etienne Gregoire Grossmann

Mianwei Zhou

Jeremy A Grata

Jeffrey Todd Daiker

Liang Wang

Robert Dale Tekolste

Yang Yang

Jian Hu

David S Koch

Daniel Roger Dominguez

Jay Pujara

Ke Wei

Peng Liu

Jian J Xu

Yu Zou

Xin Fan

David Hikaru Nakayama

Anirban Kundu

Jing Zhu

Xianyu Zhao

Chunming Wang

Bruce Hui-der Chu

Yefei Peng

Miles A Libbey

Yida Zhao

Liu Liu

Amir Radman

Jr Daly

Timothy Patrick Daly Jr

Jr Timothy Patrick Daly

Techniques are described for scenario data generation. An example method can include processing location-specific sensor information of a traffic event at a location on a road network. The method can include determining a location-specific element based on the location-specific sensor information, the location-specific element associated with a flow of traffic at a time point. The method can further include generating a location-specific model of the traffic event based on the flow of traffic and the time point. The method can further include determining, by the computing system, control instructions for a location-specific driving maneuver for an autonomous vehicle to mitigate the traffic event based on the location-specific model. The method can further include causing the control instructions to be transmitted to an actuator of the autonomous vehicle to perform the location-specific driving maneuver thereby controlling the autonomous vehicle at the location based on the model.

Techniques for autonomous vehicle event mitigation

The present teaching relates to system, method, medium for in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data acquired continuously by a plurality of types of sensors deployed on the vehicle are first received, where the plurality of types of sensor data provide information about surrounding of the vehicle. Based on at least one model, one or more items are tracked from a first of the plurality of types of sensor data acquired by one or more of a first type of the plurality of types of sensors, wherein the one or more items appear in the surrounding of the vehicle. At least some of the one or more items are then automatically labeled on-the-fly via either cross modality validation or cross temporal validation of the one or more items and are used to locally adapt, on-the-fly, the at least one model in the vehicle.

Method and system for distributed learning and adaptation in autonomous driving vehicles

Enhanced eye-tracking techniques for augmented or virtual reality display systems. An example method includes obtaining an image of an eye of a user of a wearable system, the image depicting glints on the eye caused by respective light emitters, wherein the image is a low dynamic range (LDR) image; generating a high dynamic range (HDR) image via computation of a forward pass of a machine learning model using the image; determining location information associated with the glints as depicted in the HDR image, wherein the location information is usable to inform an eye pose of the eye.

Enhanced eye tracking techniques based on neural network analysis of images

Embodiments provide image display systems and methods for a camera calibration using a two-sided diffractive optical element (DOE). More specifically, embodiments are directed to determining intrinsic parameters of a camera using a single image obtained using a two-sided DOE. The two-sided DOE has a first pattern on a first surface and a second pattern on a second surface. Each of the first and second patterns may be formed by repeating sub-patterns that are lined when tiled on each surface. The patterns on the two-sided DOE are formed such that the brightness of the central intensity peak on the image of the image pattern formed by the DOE is reduced to a predetermined amount.

Camera intrinsic calibration

Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising detecting an object in sensor data captured in an environment; detecting a marker in the sensor data captured in the environment; and determining a first location of the object in the environment based on a relative location of the object to the marker and a second location of the marker.

Detection of on-road object position using markers

A safety case discovery system includes a scenario framework and safety protocols for edge cases. The safety case discovery system receives sensor data generated by at least one sensor during operation of a vehicle and stores the sensor data in a data warehouse. The data warehouse can be queried based on a predefined scenario description to produce a subset of records which are ranked based on a relevancy of the records to the predefined scenario description. The safety case discovery system deduplicates the ranked results to produce edge cases and updates the safety protocol. The safety case discovery system can train a machine learning model vehicle based on the edge cases, to produce a trained machine learning model for optimizing the driving system.

Methods and apparatus for natural language-based safety case discovery to train a machine learning model for a driving system

The present teaching relates to method, system, medium, and implementation of a global model update center. At least one model is established at the model update center for detecting objects surrounding each of autonomous driving vehicles of a fleet. A plurality of labeled data items are received, from the fleet of autonomous driving vehicles, where each of the labeled data items is detected, based on the at least one model, from sensor data characterizing surroundings of the autonomous driving vehicles. The labeled data items are generated automatically on-the-fly by the autonomous driving vehicles. Based on the received labeled data items, at least some of the models are updated and model update information is accordingly generated. Such generated model update information is then distributed to the fleet of autonomous driving vehicles.

Method and system for closed loop perception in autonomous driving vehicles

The present teaching relates to method, system, medium, and implementation of lane planning in an autonomous vehicle. Sensor data are received that capture ground images of a road the autonomous vehicle is on. Based on the sensor data, a current lane of the road that autonomous vehicle is currently occupying is detected. Information indicating presence of a passenger in the vehicle is obtained and used to retrieve a personalized lane control model related to the passenger. Lane control for the autonomous vehicle is planned based on the detected current lane and self-aware capability parameters in accordance with the personalized lane control model. The self-aware capability parameters are used to predict operational capability of the autonomous vehicle with respect to a current location of the autonomous vehicle. The personalized lane control model is generated based on recorded human driving data.

Method and system for personalized driving lane planning in autonomous driving vehicles

The present teaching relates to method, system, medium, and implementation of lane planning in an autonomous vehicle. Sensor data are received that capture ground images of a road the autonomous vehicle is on. Based on the sensor data, a current lane of the road that autonomous vehicle is currently occupying is detected. Lane control for the autonomous vehicle is planned based on the detected current lane and self-aware capability parameters in accordance with a driving lane control model. The self-aware capability parameters are used to predict operational capability of the autonomous vehicle with respect to a current location of the autonomous vehicle. The driving lane control model is generated based on recorded human driving data to achieve human-like lane control behavior in different scenarios.

Method and system for human-like driving lane planning in autonomous driving vehicles

Augmented reality glasses include near eye displays the include sources of imagewise amplitude modulated light optical coupled to spatial phase modulators or active zone plate modulators and optically coupled to eye coupling optics. The sources of imagewise amplitude modulated light can include emissive 2D display panels or light sources coupled to imagewise amplitude modulators. The eye coupling optics can include volume holographic diffraction gratings.

Near eye 3D display with separate phase and amplitude modulators

The present teaching relates to method, system, medium, and implementation of in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data are acquired continuously via a plurality of types of sensors deployed on the vehicle, where the plurality of types of sensor data provide information about surrounding of the vehicle. One or more items surrounding the vehicle are tracked, based on some models, from a first of the plurality of types of sensor data from a first type of the plurality of types of sensors. A second of the plurality of types of sensor data are obtained from a second type of the plurality of sensors and are used to generate validation base data. Some of the one or more items are labeled, automatically, via validation base data to generate labeled at least some item, which is to be used to generate model updated information for updating the at least one model.

Method and system for on-the-fly object labeling via cross modality validation in autonomous driving vehicles

Embodiments provide image display systems and methods for one or more camera calibration using a two-sided diffractive optical element (DOE). More specifically, embodiments are directed to determining intrinsic parameters of one or more cameras using a single image obtained using a two-sided DOE. The two-sided DOE has a first pattern on a first surface and a second pattern on a second surface. Each of the first and second patterns may be formed by repeating sub-patterns that are lined when tiled on each surface. The patterns on the two-sided DOE are formed such that the brightness of the central intensity peak on the image of the image pattern formed by the DOE is reduced to a predetermined amount.

The present teaching relates to generating an alert in a vehicle. First information indicating an upcoming switch in an operating mode of the vehicle is received, which specifies a set of tasks, arranged in an order, to be completed by a driver in the vehicle to achieve the upcoming switch, and a task duration for each of the set of tasks by which the task is to be completed. A current state of the driver is obtained and used to determine a set of warnings to alert the driver to perform the set of tasks. Each warning corresponds to a task in the set of tasks and is created based on the current state of the driver. A warning schedule is generated based on the set of warnings in the order of the set of tasks and transmitted so that warnings in the warning schedule are delivered to the driver.

Method and system for augmented alerting based on driver's state in hybrid driving

The present teaching relates to method, system, and medium, for generating an augmented alert in a hybrid vehicle. First information indicating an upcoming switch in an operating mode of the vehicle is received, which specifies a set of tasks, arranged in an order, to be completed by a driver in the vehicle to achieve the upcoming switch, and a task duration for each of the set of tasks by which the task is to be completed. A current state of the driver is obtained and used to determine a set of warnings to alert the driver to perform the set of tasks. Each warning corresponds to a task in the set of tasks and is created based on the current state of the driver. A warning schedule is generated based on the set of warnings in the order of the set of tasks and transmitted so that warnings in the warning schedule are delivered to the driver.

Growing community of inventors

Saratoga, CA, United States of America

Hao Zheng