Plusai Limited

Oath Inc.

Timothy Patrick Daly, Jr

Hao Zheng

David Wanqian Liu

Anurag Ganguli

Mayank Gupta

Amit Kumar

Gael Gurvan Colas

Inderjot Singh Saggu

Anurag Paul

Ankur Gupta

Lichun Yang

Chao Tan

Robert Joseph Dingli

Ankur Agarwal

Wei-Cheng Lai

Su-Lin Wu

Scott Roy

Evgeniy Makeev

Huan Yang Chang

William Robert Pentney

Jeff Budzinski

Annie Cheng

Wenbin Wang

The present teaching relates to system, method, medium for in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data acquired continuously by a plurality of types of sensors deployed on the vehicle are first received, where the plurality of types of sensor data provide information about surrounding of the vehicle. Based on at least one model, one or more items are tracked from a first of the plurality of types of sensor data acquired by one or more of a first type of the plurality of types of sensors, wherein the one or more items appear in the surrounding of the vehicle. At least some of the one or more items are then automatically labeled on-the-fly via either cross modality validation or cross temporal validation of the one or more items and are used to locally adapt, on-the-fly, the at least one model in the vehicle.

Method and system for distributed learning and adaptation in autonomous driving vehicles

The present teaching relates to method, system, medium, and implementation of a global model update center. At least one model is established at the model update center for detecting objects surrounding each of autonomous driving vehicles of a fleet. A plurality of labeled data items are received, from the fleet of autonomous driving vehicles, where each of the labeled data items is detected, based on the at least one model, from sensor data characterizing surroundings of the autonomous driving vehicles. The labeled data items are generated automatically on-the-fly by the autonomous driving vehicles. Based on the received labeled data items, at least some of the models are updated and model update information is accordingly generated. Such generated model update information is then distributed to the fleet of autonomous driving vehicles.

Method and system for closed loop perception in autonomous driving vehicles

In some embodiments, a method comprises receiving, at a processor of an autonomous vehicle and from at least one sensor, sensor data distributed within a time window. A first event being a first event type occurring at a first time in the time window is identified by the processor using a software model based on the sensor data. At least one first attribute associated with the first event is extracted by the processor. A second event being the first event type occurring at a second time in the time window is identified by the processor based on the at least one first attribute. In response to determining that the second event is not yet recognized as being the first event type, a first label for the second event is generated by the processor.

Methods and apparatus for automatically labeling data processing events in autonomous driving vehicles via machine learning

The present teaching relates to method, system, medium, and implementation of lane planning in an autonomous vehicle. Sensor data are received that capture ground images of a road the autonomous vehicle is on. Based on the sensor data, a current lane of the road that autonomous vehicle is currently occupying is detected. Information indicating presence of a passenger in the vehicle is obtained and used to retrieve a personalized lane control model related to the passenger. Lane control for the autonomous vehicle is planned based on the detected current lane and self-aware capability parameters in accordance with the personalized lane control model. The self-aware capability parameters are used to predict operational capability of the autonomous vehicle with respect to a current location of the autonomous vehicle. The personalized lane control model is generated based on recorded human driving data.

Method and system for personalized driving lane planning in autonomous driving vehicles

The present teaching relates to method, system, medium, and implementation of lane planning in an autonomous vehicle. Sensor data are received that capture ground images of a road the autonomous vehicle is on. Based on the sensor data, a current lane of the road that autonomous vehicle is currently occupying is detected. Lane control for the autonomous vehicle is planned based on the detected current lane and self-aware capability parameters in accordance with a driving lane control model. The self-aware capability parameters are used to predict operational capability of the autonomous vehicle with respect to a current location of the autonomous vehicle. The driving lane control model is generated based on recorded human driving data to achieve human-like lane control behavior in different scenarios.

Method and system for human-like driving lane planning in autonomous driving vehicles

The present teaching relates to method, system, medium, and implementation of in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data are acquired continuously via a plurality of types of sensors deployed on the vehicle, where the plurality of types of sensor data provide information about surrounding of the vehicle. One or more items surrounding the vehicle are tracked, based on some models, from a first of the plurality of types of sensor data from a first type of the plurality of types of sensors. A second of the plurality of types of sensor data are obtained from a second type of the plurality of sensors and are used to generate validation base data. Some of the one or more items are labeled, automatically, via validation base data to generate labeled at least some item, which is to be used to generate model updated information for updating the at least one model.

Method and system for on-the-fly object labeling via cross modality validation in autonomous driving vehicles

Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising determining map data and detection data for an area in an environment; determining a change in the area based on the map data and the detection data; and generating control data based on the change.

Mapping and detection for safe navigation

A system includes an electronic control unit (ECU) having a housing and configured to be disposed within an interior of an autonomous vehicle. The system also includes an inertial measurement unit (IMU) rigidly disposed with the housing of the ECU. The system also includes a processor configured to receive measurement data from the IMU. The IMU and the processor is powered by a battery separate from a power source of the ECU. The processor is configured to identify a tamper event of the ECU based on the measurement data from the IMU. The processor is also configured to send a signal to cause a tamper response of the tamper event.

Methods and apparatus for tamper detection of a vehicle system

Techniques are described for determining whether to process a job request. An example, method can include a device emitting a first pulse using a light detection and ranging (LIDAR) system coupled to an autonomous vehicle. The device can receive a first signal reflected off of an object. The device can emit a second pulse using the system, a threshold time interval being configured for the second laser pulse to hit the object in motion. The device can receive a second signal reflected off of the object. The device can determine a first time of flight information of the first signal and a second time of flight information of the second signal. The device can determine a velocity of the object based at least in part on the first time of flight information and the second time of flight information.

Velocity estimation using light detection and ranging (LIDAR) system

The present teaching relates to generating an alert in a vehicle. First information indicating an upcoming switch in an operating mode of the vehicle is received, which specifies a set of tasks, arranged in an order, to be completed by a driver in the vehicle to achieve the upcoming switch, and a task duration for each of the set of tasks by which the task is to be completed. A current state of the driver is obtained and used to determine a set of warnings to alert the driver to perform the set of tasks. Each warning corresponds to a task in the set of tasks and is created based on the current state of the driver. A warning schedule is generated based on the set of warnings in the order of the set of tasks and transmitted so that warnings in the warning schedule are delivered to the driver.

Method and system for augmented alerting based on driver's state in hybrid driving

The present teaching relates to method, system, and medium, for generating an augmented alert in a hybrid vehicle. First information indicating an upcoming switch in an operating mode of the vehicle is received, which specifies a set of tasks, arranged in an order, to be completed by a driver in the vehicle to achieve the upcoming switch, and a task duration for each of the set of tasks by which the task is to be completed. A current state of the driver is obtained and used to determine a set of warnings to alert the driver to perform the set of tasks. Each warning corresponds to a task in the set of tasks and is created based on the current state of the driver. A warning schedule is generated based on the set of warnings in the order of the set of tasks and transmitted so that warnings in the warning schedule are delivered to the driver.

A system includes sensors onboard an autonomous vehicle, a processor, and a memory. The memory stores instructions for the processor to receive a first image pair from the sensors at a first time and a second image pair from the sensors at a second time. Each image pair includes at least one static feature in an environment of the autonomous vehicle. The memory also stores instructions to determine a distance travelled by the autonomous vehicle between the first and second times, and to determine a correction to a disparity map based on (1) a first disparity associated with the static feature(s) and the first image pair, (2) a second disparity associated with the static feature(s) and the second image pair, and (3) the distance travelled. The memory also stores instructions to cause the correction to be applied to the disparity map.

Methods and apparatus for calibrating stereo imagery using motion of vehicle

The present teaching relates to a method, system, medium, and implementation of processing image data in an autonomous driving vehicle. Sensor data acquired by one or more types of sensors deployed on the vehicle are continuously received. The sensor data provide different information about surrounding of the vehicle. Based on a first data set acquired by a first sensor of a first type of the one or more types of sensors at a specific time, an object is detected, where the first data set provides a first type of information about the surrounding of the vehicle. Depth information of the object is then estimated via object centric stereo at object level based on the object detected as well as a second data set acquired by a second sensor of the first type of the one or more types of sensors at the specific time. The second data set provides the first type of information about the surrounding of the vehicle with a different perspective as compared with the first data set.

Method and system for object centric stereo in autonomous driving vehicles

In an embodiment, a processor is configured to perform, while a vehicle is driving in an uncontrolled environment, a self-calibration routine for each sensor from the plurality of sensors to determine at least one calibration parameter value associated with that sensor. The processor is further configured to determine, while the vehicle is driving in the uncontrolled environment, and automatically in response to performing the self-calibration routine, that at least one sensor from the plurality of sensors has moved and/or is inoperative based on the at least one calibration parameter value associated with the at least one sensor being outside a predetermined acceptable range. The processor is further configured to perform, in response to determining that at least one sensor from the plurality of sensors has moved and/or is inoperative, at least one remedial action at the vehicle while the vehicle is driving in the uncontrolled environment.

Operating a vehicle in response to detecting a faulty sensor using calibration parameters of the sensor

In some embodiments, a method includes receiving a first image and a second image from a stereo camera pair. The method includes selecting a first row of pixels from the rectified image and a set of rows of pixels from the second image and comparing the first row of pixels with each row of pixels from the set of rows of pixels to determine disparity values. The method includes determining a pair of rows of pixels having the first row of pixels and a second row of pixels from the set of rows of pixels. The pair of rows of pixels has an offset no greater than an offset between the first row of pixels and each row of pixels from remaining rows of pixels. The method includes adjusting, based on the offset, the relative rotational position between the first stereo camera and the second stereo camera.

Growing community of inventors

San Jose, CA, United States of America

Timothy Patrick Daly, Jr