Nio Usa, Inc.

Anthony Tao Liang

Guan Wang

Prashant Jain

Vidya Elangovan

Gautam Muralidhar

Nadav Aharony

Fei Xiao

Abhishek Singhal

Divya Ramakrishnan

Christopher Francis Pouliot

James S Higgins

Rachad Youssef Gamarra

Lisa E Falkson

Infantdani Abel Rayan

Annie Jui Shan Chang

Joshua T Butler

Embodiments of the present disclosure are directed to providing a more accurate presentation of a vehicle's surroundings when the presentation combines images from multiple cameras. Cameras may capture the same portion of the environment. If an object is within view of at least two cameras, each camera produces a portion of an overall view, such as a 360 degree synthetic top-down view. The seam, angle and/or geometry, angle between any two images is dynamically determined, such as due to proximity (or lack thereof) to an object. As a result the synthetic top-down view may present an image of an object more prominently and/or avoid having the seam between camera images fall on the image of the object, which may otherwise result in the image of the object being omitted from both camera images.

Dynamic seam adjustment of image overlap zones from multi-camera source images

Embodiments of the present disclosure are directed to providing an Augmented Reality (AR) navigation display in a vehicle. More specifically, embodiments are directed to rendering AR indications of a navigation route over a camera video stream in perspective. According to one embodiment, visual tracking can be performed on features in the video data and camera pose, i.e., a matrix encapsulating position and orientation, can be determined for each frame of video based on both the visual tracking and navigation sensor data. These separately determined camera poses can then be merged or fused into a single camera pose that is more accurate and more stable and which can then be used in rendering more realistic AR route indicators onto the video of the real-world route captured by the camera.

Sensor fusion methods for augmented reality navigation

Embodiments of the present disclosure are directed to an augmented reality based user's manual for a vehicle implemented as an application on a mobile device which allows the user to point a mobile phone, tablet or an augmented reality headset at any part of the vehicle interior or exterior and experience augmented annotations, overlays, popups, etc. displayed on images of real parts of the car captured by the user's mobile device. Embodiments provide for estimating the camera pose in six degrees of freedom based on the content of the captured image or video and using a neural network trained on a dense sampling of a three-dimensional model of the car rendered with realistic textures to identify and properly align the augmented reality presentation with the image of the vehicle being captured by the mobile device.

Camera pose estimation method for augmented reality manual for cars

A method and system for authenticating a user of a vehicle are described. In one embodiment, a user profile is associated with a user of a vehicle. The user profile is used to compare a sample of the voice of the user with a database onboard the vehicle as well as a database stored on a network connected server in communication with the vehicle. A user profile may be associated with a number of vehicles. Users may have a number of levels of permissions associated with a number of different vehicles. A user may be enabled to access a number of vehicle features by providing a voice sample via microphones in and around the vehicle. A voiceprint associated with the user may be improved based upon audio collected by microphones in and around the vehicle.

Voice biometric pre-purchase enrollment for autonomous vehicles

Embodiments are directed to employing various methods and devices to access a vehicle and controlling a level of access granted based on the device and methods employed. Controlling access to a vehicle can comprise maintaining a plurality of security profiles, each security profile associated with a different access device or type of access device. When the presence of an access device in proximity to the vehicle or a request to access the vehicle is detected, a security profile associated with the detected access device or the request to access the vehicle can be retrieved and used to determine whether to grant access to the vehicle. In response to determining to grant access to the vehicle, access to the vehicle can be provided according to an access level for the detected access device or the request to access the vehicle identified in the retrieved security profile.

Controlling access to a vehicle using wireless access devices

An automated system for controlling a vehicle gathers a set of sensor information for a series of driving actions that comprises a driving path of an individual driver. A context is determined that is related to the set of sensor information for the series of driving actions that comprise the driving path of the individual driver. For example, the context may be a weather condition or when the driver merges into traffic. A personalization score is determined for the series of driving actions for the individual driver based on the context. A personalization score is determined for one or more other driving paths and a driving path with the highest personalization score is identified. The driving path with the highest personalization score is then chosen for automated driving.

Machine learning for personalized driving

The systems and methods described herein can be applied to a vehicle equipped with a sensor suite that can observe information about a location, for example, a traffic light duration. The system can record information, e.g., the traffic light colors, duration of color changes, and location of the traffic lights and can upload this information to the cloud. Then, the system can augment or learn about the location, e.g., learning of traffic patterns, and store the augmented data as database-based information, where available. The learned information can help a requesting vehicle to better estimate an estimated time of arrival (ETA) for common routes taken, provide more accurate ETAs based on historical knowledge, and/or calculate or provide alternative route information.

Growing community of inventors

Palo Alto, CA, United States of America

Anthony Tao Liang