Plusai Limited

Mianwei Zhou

Yibo Chen

Inderjot Singh Saggu

Hao Zheng

Ankur Agarwal

David Wanqian Liu

Anurag Ganguli

Anurag Paul

Abhishek Bhatia

Kuan-Wei Liu

In some embodiments, a method can include executing a first machine learning model to detect at least one lane in each image from a first set of images. The method can further include determining an estimate location of a vehicle for each image, based on localization data captured using at least one localization sensor disposed at the vehicle. The method can further include selecting lane geometry data for each image, from a map and based on the estimate location of the vehicle. The method can further include executing a localization model to generate a set of offset values for the first set of images based on the lane geometry data and the at least one lane in each image. The method can further include selecting a second set of images from the first set of images based on the set of offset values and a previously-determined offset threshold.

Methods and apparatus for automatic collection of under-represented data for improving a training of a machine learning model

A method comprises: receiving, at a processor, a first image from a first camera from a stereo camera pair and a second image from a second camera from the stereo camera pair. The method also includes determining, at the processor using a machine learning model, a first set of objects in the first image. The processor determines an object type. The processor identifies a second set of objects in the second image associated with the first plurality of objects. The method also includes calculating, at the processor, a set of disparity values between the first image and the second image based on (1) an object from the first set of objects, (2) an object from the second set of objects and associated with the object from the first set of objects, and (3) an object type of the object from the first set of objects.

Methods and apparatus for depth estimation using stereo cameras in a vehicle system

This application is directed to aerial view generation for vehicle control. A vehicle obtains a forward-facing view of a road captured by a front-facing camera of a vehicle and applies a machine learning model to process the forward-facing view to predict determine a trajectory of the vehicle and a road layout based on a Frenet-Serret coordinate system of the road for the vehicle. The trajectory of the vehicle is combined with the road layout to predict an aerial view of the road, and the aerial view of the road is used to at least partially autonomously drive the vehicle. In some embodiments, the machine learning model is applied to process the forward-facing view to determine a first location of an obstacle vehicle in the Frenet-Serret coordinate system. The obstacle vehicle is placed on the aerial map based on the first location of the obstacle vehicle.

Vehicle placement on aerial views for vehicle control

This application is directed to augmenting training data used for vehicle driving modelling. A computer system obtains a first image of a road and identifies a drivable area of the road within the first image. The computer system obtains an image of an object and generates a second image from the first image by overlaying the image of the object over the drivable area. The second image is added to a corpus of training images to be used by a machine learning system to generate a model for facilitating driving of a vehicle (e.g., at least partial autonomously). In some embodiments, the computer system applies machine learning to train a model using the corpus of training images and distributes the model to one or more vehicles. In use, the model processes road images captured by the one or more vehicles to facilitate vehicle driving.

Data augmentation for vehicle control

In an embodiment, a method comprises detecting, at a processor of an autonomous vehicle, a discrepancy between a map and a property sensed by at least one sensor onboard the autonomous vehicle, the property being associated with an external environment of the autonomous vehicle. In response to detecting the discrepancy, and based on the discrepancy, an annotation for the map is generated via the processor. A signal representing the annotation is caused to be transmit to a compute device that is remote from the autonomous vehicle. A signal representing a map update is received from the compute device that is remote form the autonomous vehicle. The map update is generated based on the annotation, the map update (1) including replacement information for a region of the map associated with the annotation, and (2) not including replacement information for a remainder of the map.

Methods and apparatus for navigating an autonomous vehicle based on a map updated in regions

This application is directed to aerial view generation for vehicle control. A vehicle obtains a forward-facing view of a road captured by a front-facing camera of a vehicle and applies a machine learning model to process the forward-facing view to predict determine a trajectory of the vehicle and a road layout based on a Frenet-Serret coordinate system of the road for the vehicle. The trajectory of the vehicle is combined with the road layout to predict an aerial view of the road, and the aerial view of the road is used to at least partially autonomously drive the vehicle. In some embodiments, the machine learning model is applied to process the forward-facing view to determine a first location of each of an obstacle vehicle in the Frenet-Serret coordinate system. The first location of the obstacle vehicle is converted to a vehicle location on the aerial view of the road.

Aerial view generation for vehicle control

Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising determining lane detection data and object detection data associated with an environment; determining a lane template based on the lane detection data; and generating localization data that identifies a location of an object in the environment based on the lane template and the object detection data.

Vehicle localization based on lane templates

A user-generated graphical representation can be sent into a generative network to generate a synthetic image of an area including a road, the user-generated graphical representation including at least three different colors and each color from the at least three different colors representing a feature from a plurality of features. A determination can be made that a discrimination network fails to distinguish between the synthetic image and a sensor detected image. The synthetic image can be sent, in response to determining that the discrimination network fails to distinguish between the synthetic image and the sensor-detected image, into an object detector to generate a non-user-generated graphical representation. An objective function can be determined based on a comparison between the user-generated graphical representation and the non-user-generated graphical representation. A perception model can be trained using the synthetic image in response to determining that the objective function is within a predetermined acceptable range.

Navigating a vehicle based on data processing using synthetically generated images

Embodiments provide improved techniques for identifying object and vehicle locations on an HD map using camera images. Synthetic data may be generated by populating a high-definition (HD) map with an object. The populated HD map can be projected to a two-dimensional camera view image depicting the object. The HD map and object detection data generated from image (e.g., identifying lane/object locations within the image) can be used to train the model to identify HD map locations of an autonomous vehicle capturing images as it travels, as well as various objects detected from those captured images. Subsequently, a new image may be processed using object detection techniques to detect lane/object locations within the image. An HD map and the detected lane/object data may be provided to the model, which in turn identifies, on the HD map, the locations of the vehicle and the various objects.

Model-based localization on high-definition maps

This application is directed to on-vehicle behavior modeling of vehicles. A vehicle has one or more processors, memory, a plurality of sensors, and a vehicle control system. The vehicle collects training data via the plurality of sensors, and the training data include data for one or more vehicles during a collection period. The vehicle locally applies machine learning to train a vehicle driving behavior model using the collected training data. The vehicle driving behavior model is configured to predict a behavior of one or more vehicles. The vehicle subsequently collecting sensor data from the plurality of sensors and drives the vehicle by applying the vehicle driving behavior model to predict vehicle behavior based on the collected sensor data. The vehicle driving behavior model is configured to predict behavior of an ego vehicle and/or a distinct vehicle that appears near the ego vehicle.

On-vehicle driving behavior modelling

The present teaching relates to method, system, medium, and implementation of human-like vehicle control for an autonomous vehicle. Information related to a target motion to be achieved by the autonomous vehicle is received, wherein the information includes a current vehicle state of the autonomous vehicle. A first vehicle control signal is generated with respect to the target motion and the given vehicle state in accordance with a vehicle kinematic model. A second vehicle control signal is generated in accordance with a human-like vehicle control model, with respect to the target motion, the given vehicle state, and the first vehicle control signal, wherein the second vehicle control signal modifies the first vehicle control signal to achieve human-like vehicle control behavior.

Growing community of inventors

Santa Clara, CA, United States of America

Mianwei Zhou