Uatc, LLC

Aurora Operations, Inc.

Uber Technologies, Inc.

Nemanja Djuric

Vladan Radosavljevic

Fang-Chieh Chou

Jeff Schneider

Henggang Cui

Tsung-Han Lin

Tzu-Kuo Huang

Thi Duong Nguyen

Sai Bhargav Yalamanchi

Abhishek Mohta

Galen Clark Haynes

Shivam Gautam

Carlos Vallespi-Gonzalez

Brian C Becker

Junheng Wang

Mohana Prasad Sathya Moorthy

David McAllister Bradley

Anthony Joseph Stentz

Micol Marchetti-Bowick

Tian Lan

Joseph Lawrence Amato

Alexander David Styler

Neil Traft

Niels Joubert

Brian Baumgartner

Chi-Kuei Liu

Peter M Kingston

John Houston

Huahua Wang

Ian Dewancker

A teleassist system may be used to assist with detecting and responding to an emergency vehicle present within an environment of an autonomous vehicle. The teleassist system can help confirm whether the emergency vehicle is active or inactive based on evaluating sensor data that is captured by one or more sensors of the autonomous vehicle and that is associated with the emergency vehicle. For example, the teleassist system can provide a teleassist operator input confirming whether a detected vehicle is an emergency vehicle based on evaluation of various aspects, including siren, vehicle shape, vehicle color, and/or flashing light emitted by flash device(s) of the vehicle, etc. The teleassist operator input can further include one or more suggested autonomous vehicle control operations that enable the autonomous vehicle to comply with detection of an emergency vehicle.

Emergency vehicle detection for autonomous vehicle

An autonomous platform can obtain sensor data descriptive of an actor in an environment of an autonomous vehicle and at least a portion of the environment of the autonomous vehicle that does not include the actor, the sensor data comprising at least one sweep of the environment of the autonomous vehicle; process the sensor data with a machine-learned perception model to generate a detection of the actor and one or more predicted future velocities; and determine a motion trajectory for the autonomous vehicle based at least in part on the detection and the one or more predicted future velocities.

Perception system for an autonomous vehicle

Systems, methods, tangible non-transitory computer-readable media, and devices for autonomous vehicle operation are provided. For example, a computing system can receive object data that includes portions of sensor data. The computing system can determine, in a first stage of a multiple stage classification using hardware components, one or more first stage characteristics of the portions of sensor data based on a first machine-learned model. In a second stage of the multiple stage classification, the computing system can determine second stage characteristics of the portions of sensor data based on a second machine-learned model. The computing system can generate an object output based on the first stage characteristics and the second stage characteristics. The object output can include indications associated with detection of objects in the portions of sensor data.

Multiple stage image based object detection and recognition

Various examples are directed to systems and methods for controlling an autonomous vehicle comprising a tractor and a trailer. For example, a system may determine that a line from a position of a first sensor on the autonomous vehicle to a position of a first actor in an environment of the autonomous vehicle intersects the trailer. The system may determine that the first actor is in a blind spot of the autonomous vehicle, generate a motion plan for the autonomous vehicle, and control the autonomous vehicle in accordance with the motion plan.

Autonomous vehicle blind spot management

Systems and methods are disclosed for detecting and predicting the motion of objects within the surrounding environment of a system such as an autonomous vehicle. For example, an autonomous vehicle can obtain sensor data from a plurality of sensors comprising at least two different sensor modalities (e.g., RADAR, LIDAR, camera) and fused together to create a fused sensor sample. The fused sensor sample can then be provided as input to a machine learning model (e.g., a machine learning model for object detection and/or motion prediction). The machine learning model can have been trained by independently applying sensor dropout to the at least two different sensor modalities. Outputs received from the machine learning model in response to receipt of the fused sensor samples are characterized by improved generalization performance over multiple sensor modalities, thus yielding improved performance in detecting objects and predicting their future locations, as well as improved navigation performance.

Systems and methods for sensor data processing and object detection and motion prediction for robotic platforms

Systems, methods, tangible non-transitory computer-readable media, and devices associated with trajectory prediction are provided. For example, trajectory data and goal path data can be accessed. The trajectory data can be associated with an object's predicted trajectory. The predicted trajectory can include waypoints associated with waypoint position uncertainty distributions that can be based on an expectation maximization technique. The goal path data can be associated with a goal path and include locations the object is predicted to travel. Solution waypoints for the object can be determined based on application of optimization techniques to the waypoints and waypoint position uncertainty distributions. The optimization techniques can include operations to maximize the probability of each of the solution waypoints. Stitched trajectory data can be generated based on the solution waypoints. The stitched trajectory data can be associated with portions of the solution waypoints and the goal path.

Trajectory prediction for autonomous devices

Systems and methods for predicting object motion and controlling autonomous vehicles are provided. In one example embodiment, a computer implemented method includes obtaining state data indicative of at least a current or a past state of an object that is within a surrounding environment of an autonomous vehicle. The method includes obtaining data associated with a geographic area in which the object is located. The method includes generating a combined data set associated with the object based at least in part on a fusion of the state data and the data associated with the geographic area in which the object is located. The method includes obtaining data indicative of a machine-learned model. The method includes inputting the combined data set into the machine-learned model. The method includes receiving an output from the machine-learned model. The output can be indicative of a plurality of predicted trajectories of the object.

Object motion prediction and autonomous vehicle control

Systems and methods for training machine-learned models are provided. A method can include receiving a rasterized image associated with a training object and generating a predicted trajectory of the training object by inputting the rasterized image into a first machine-learned model. The method can include converting the predicted trajectory into a rasterized trajectory that spatially corresponds to the rasterized image. The method can include utilizing a second machine-learned model to determine an accuracy of the predicted trajectory based on the rasterized trajectory. The method can include determining an overall loss for the first machine-learned model based on the accuracy of the predictive trajectory as determined by the second machine-learned model. The method can include training the first machine-learned model by minimizing the overall loss for the first machine-learned model.

Systems and methods for training predictive models for autonomous devices

Systems, methods, tangible non-transitory computer-readable media, and devices associated with the motion prediction and operation of a device including a vehicle are provided. For example, a vehicle computing system can access state data including information associated with locations and characteristics of objects over a plurality of time intervals. Trajectories of the objects at subsequent time intervals following the plurality of time intervals can be determined based on the state data and a machine-learned tracking and kinematics model. The trajectories of the objects can include predicted locations of the objects at subsequent time intervals that follow the plurality of time intervals. Further, the predicted locations of the objects can be based on physical constraints of the objects. Furthermore, indications, which can include visual indications, can be generated based on the predicted locations of the objects at the subsequent time intervals.

Motion prediction for autonomous devices

A transport system can manage an on-demand transportation service to connect available vehicles with users, and can compile ride history data for each user. The ride history data can indicate the contextual usage of the on-demand transportation service by the user. Based on the ride history data, the transport system can determine demographic and personal interest information of the respective user. The transport system may then personalize one or more ride characteristics of any ride requested by the user based on the demographic and personal interest information determined from the ride history of the user.

Personalizing ride experience based on contextual ride usage data

Systems and methods for predicting object motion and controlling autonomous vehicles are provided. In one example embodiment, a computer implemented method includes obtaining state data indicative of at least a current or a past state of an object that is within a surrounding environment of an autonomous vehicle. The method includes obtaining data associated with a geographic area in which the object is located. The method includes generating a combined data set associated with the object based at least in part on a fusion of the state data and the data associated with the geographic area in which the object is located. The method includes obtaining data indicative of a machine-learned model. The method includes inputting the combined data set into the machine-learned model. The method includes receiving an output from the machine-learned model. The output can be indicative of a predicted trajectory of the object.

Growing community of inventors

Pittsburgh, PA, United States of America

Nemanja Djuric