Zoox, Inc.

Juhana Kangaspunta

Dragomir Dimitrov Anguelov

William Anthony Silva

Benjamin Isaac Zwiebel

James William Vaisey Philbin

Kai Zhenyu Wang

Vasiliy Karasev

Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object. An occlusion grid representing the occluded area can be stored as map data or can be dynamically generated. An occlusion grid can include occlusion fields, which represent discrete two- or three-dimensional areas of driveable environment. An occlusion field can indicate an occlusion state and an occupancy state, determined using LIDAR data and/or image data captured by the vehicle. An occupancy state of an occlusion field can be determined by ray casting LIDAR data or by projecting an occlusion field into segmented image data. The vehicle can be controlled to traverse the environment when a sufficient portion of the occlusion grid is visible and unoccupied.

Occlusion aware planning and control

A system may automatically create training datasets for training a segmentation model to recognize features such as lanes on a road. The system may receive sensor data representative of a portion of an environment and map data from a map data store including existing map data for the portion of the environment that includes features present in that portion of the environment. The system may project or overlay the features onto the sensor data to create training datasets for training the segmentation model, which may be a neural network. The training datasets may be communicated to the segmentation model to train the segmentation model to segment data associated with similar features present in different sensor data. The trained segmentation model may be used to update the map data store, and may be used to segment sensor data obtained from other portions of the environment, such as portions not previously mapped.

Automatic creation and updating of maps

Occulsion aware planning and control

An object position and/or orientation can be determined based on image data and object contact points. Image data can be captured representing an object, such as a vehicle. Vehicle contact points can be identified in the image data representing wheel contacts with the ground. For an individual vehicle contact point (e.g., a left-front wheel of the second vehicle), a ray can be determined that emanates from the image sensor and passes through the vehicle contact point. To determine a location and velocity of the vehicle, the ray can be unprojected onto a three-dimensional surface mesh, and an intersection point between the ray and the three-dimensional surface mesh can be used as an initial estimate for the projected location of the vehicle contact point in the world. The estimated location can be adjusted based on various cost functions to optimize an accuracy of the locations of the estimated vehicle contact points.

Pose determination from contact points

Occlusion aware planning

Techniques are disclosed for filtering point cloud data associated with particulate matter (e.g., gas, exhaust, fog, etc.) which do not impact driving from data used to plan a trajectory and/or a route of a robotic platform. The filtering may be based on determining that a set of points associated with a point cloud represents a navigable space for the robotic platform. The point cloud data may be filtered based on a determination that one or more safety conditions are satisfied. Semantic segmentation may be performed on an image to determine pixel classification probability distributions associated with pixels of the image. Data associated with the set of points may be projected onto the image to identify corresponding pixels. Confidence scores associated with pixel classification probability distributions for the identified pixels may be queried. A classification probability distribution for the set of points may be determined based at least in part on the queried confidence scores.

Growing community of inventors

San Francisco, CA, United States of America

Juhana Kangaspunta