Zoox, Inc.

Jesse Sol Levinson

Ashutosh Gajanan Rege

Gabriel Thurston Sibley

Timothy David Kentley-Klay

Gary Linscott

Derek Adams

Bertrand Robert Douillard

Ravi Verma Gogna

Nitesh Shroff

Paul Orecchio

Dan Xie

Amanda Lee Kelly Lockwood

Rachad Youssef Gamara

Brice Rebsamen

Marin Kobilarov

Timothy Caldwell

Elena Stephanie Stumm

Ali Javidan

Timothy David Kentley

Amanda Blair Lind

Dragomir Dimitrov Anguelov

Christopher John Stoffel

Subhasis Das

Zeng Wang

Patrick Blaes

Daniel Miller

Aleksandrs Ecins

Ian Baldwin

Sarah Tariq

Kratarth Goel

Derek Xiang Ma

Adam Pollack

Ekaterina Hristova Taralova

Daniel Earl Meyer

Valmiki Satya Rampersad

Justin Nordgreen

James William Vaisey Philbin

Joseph Funke

Ryan McMichael

Johannes Edren

Robert J Chen

Robert Nicholas Moor

Mohammad Umar Piracha

Maxwell Yaron

Ryan Mcmichael

A light sensor and apparatus capable of determining depth and image data simultaneously from a target in an environment is discussed herein. The apparatus includes an array of photodetector elements comprising a first portion to detect frequency modulated continuous wave (FMCW) signals from the reflected light signals and one or more additional portions that detect visible light signals from the environment.

Lidar sensor with embedded photodetector

Techniques for adjusting vehicle models based on environmental conditions are discussed herein. The techniques may include receiving image data representing a portion of an environment in which a vehicle is operating and inputting the image data into a machine learned model. Additionally, data representing an environmental condition associated with the environment may be received from a sensor of the vehicle to detect changes in the environmental conditions such that one or more actions associated with the machine learned model or an output of the machine learned model may be performed. Some of the techniques may also include running multiple machine learned models or multiple configurations of a machine learned model in parallel and selecting different outputs of the machine learned model(s) to adjust for changes in the environmental conditions. For instance, individual outputs may be selected based on environmental conditions, confidence scores, thresholds, etc.

Adjusting vehicle models based on environmental conditions

Parallel vehicle model processing

An autonomous delivery vehicle including locking storage containers may be used for item deliveries, rejections, returns, and/or third-party fulfillment. A delivery vehicle or robot may include a number of locking storage containers, an authorization interface, and one or more sensors to receive delivery requests, detect and authorize users, and control locker access at various delivery locations to allow users to receive delivered items, and reject or return items. The vehicle may also include a passenger compartment to transport one or more passengers. The vehicle may be reconfigurable to accommodate different combinations of lockers and/or passenger seats. An item delivery system may receive delivery requests and determine routes for delivery vehicles, including centralized delivery locations and/or direct deliveries to recipients.

Mobile delivery system with secure access lockers

Systems, apparatus, and methods implemented in algorithms, software, firmware, logic, or circuitry may be configured to process data and sensory input in real-time to detect an object. In particular, a method may include determining a predicted object path of the object. The method may further include determining a predict point of impact between the autonomous vehicle and the object based in part on the predicted object path. The method may further include identifying a preferred point of impact that is associated with a safety system disposed on the autonomous vehicle. The method may further include causing the autonomous vehicle to perform a maneuver based in part on the preferred point of impact.

Autonomous vehicle fleet service and system

Techniques are discussed for modifying map elements associated with map data. Map data can include three-dimensional data (e.g., LIDAR data) representing an environment, while map elements can be associated with the map data to identify locations and semantic information associated with an environment, such as regions that correspond to driving lanes or crosswalks. A trajectory associated with the map data can be updated, such as when aligning one or more trajectories in response to a loop closure, updated calibration, etc. The transformation between a trajectory and an updated trajectory can be applied to map elements to warp the map elements so that they correspond to the updated map data, thereby providing automatic and accurate techniques for updating map elements associated with map data.

Modifying map elements associated with map data

Techniques for item delivery and delivery systems including autonomous delivery vehicles used to identify optimal delivery locations and/or deliver physical items to pedestrians is discussed herein. In some examples, a fleet management system may receive sensor data captured from sensor devices of vehicles of a fleet of vehicles while traversing within an environment. The fleet management system may use the sensor data to identify region(s) of the environment within which a threshold number of pedestrians may be located. Based on identifying a region having a threshold number of pedestrians, the fleet management system may determine how many vehicles of the fleet of vehicles are available to render delivery services to the pedestrians within the region. In some examples, the fleet management system may identify an available vehicle of the fleet of vehicles to which instructions may be sent. Such instructions may include transporting items to pedestrians within the region.

Fleet management for providing goods to pedestrians

Techniques for item delivery including autonomous delivery vehicles used to identify optimal delivery locations and/or deliver physical items to pedestrians are discussed herein. In some examples, a vehicle may receive the instructions from a fleet management system. The vehicle may determine that the instructions cause the vehicle to deliver a quantity of goods to a specified region within the environment. Based on the instructions, the vehicle may transition to a delivery vehicle and navigate to the region. Upon arriving to the region, the vehicle may receive sensor data captured from sensor devices of the vehicle. Such sensor data may be used to detect a plurality of pedestrians within the region. Based on identifying the plurality of pedestrians within the region, the vehicle may determine a dispensing strategy. In some examples, the vehicle may deliver the goods to the plurality of pedestrians within the region according to the dispensing strategy.

Dynamic vehicle routing for providing goods to pedestrians

Mobile delivery vehicle management and routing

Perception sensors of a vehicle can be used for various operating functions of the vehicle. A computing device may receive sensor data from the perception sensors, and may calibrate the perception sensors using the sensor data, to enable effective operation of the vehicle. To calibrate the sensors, the computing device may project the sensor data into a voxel space, and determine a voxel score comprising an occupancy score and a residual value for each voxel. The computing device may then adjust an estimated position and/or orientation of the sensors, and associated sensor data, from at least one perception sensor to minimize the voxel score. The computing device may calibrate the sensor using the adjustments corresponding to the minimized voxel score. Additionally, the computing device may be configured to calculate an error in a position associated with the vehicle by calibrating data corresponding to a same point captured at different times.

Sensor perturbation

A method for autonomously operating a driverless vehicle along a path between a first geographic location and a destination may include receiving communication signals from the driverless vehicle. The communication signals may include sensor data from the driverless vehicle and data indicating occurrence of an event associated with the path. The communication signals may also include data indicating that a confidence level associated with the path is less than a threshold confidence level due to the event. The method may also include determining, via a teleoperations system, a level of guidance to provide the driverless vehicle based on data associated with the communication signals, and transmitting teleoperations signals to the driverless vehicle. The teleoperations signals may include guidance to operate the driverless vehicle according to the determined level of guidance, so that a vehicle controller maneuvers the driverless vehicle to avoid, travel around, or pass through the event.

Interactions between vehicle and teleoperations system

Various embodiments relate generally to autonomous vehicles and associated mechanical, electrical and electronic hardware, computer software and systems, and wired and wireless network communications to provide an autonomous vehicle fleet as a service. In particular, a method may include monitoring a fleet of vehicles, at least one of which is configured to autonomously transit from a first geographic region to a second geographic region, detecting data indicating an event associated with the vehicle having a calculated confidence level, receiving data representing a subset of candidate trajectories responsive to detecting the event, which is associated with a planned path for the vehicle, identifying guidance data to select from one or more of the candidate trajectories as a guided trajectory, receiving data representing a selection of a candidate trajectory, and transmitting the selection of the candidate trajectory as of the guided trajectory to the vehicle.

Techniques are discussed for determining a data level for portions of data for processing. In some cases, a data level can correspond to a resolution level, a compression level, a bit rate, and the like. In the context of image data, the techniques can determine a region of first image data to be processed a high resolution and a region of second image data to be processed at a low resolution. The regions can be determined by a machine learned algorithm that is trained to output identifications of such regions. Training data may be determined by identifying differences in outputs based on the first and second image data. The image data associated with the determined regions and the determined resolutions can be processed to perform object detection, classification, segmentation, bounding box generation, and the like, thereby conserving processing, bandwidth, and/or memory resources in real time systems.

Optimizing data levels for processing, transmission, or storage

Systems, methods, and apparatuses described herein are directed to performing segmentation on voxels representing three-dimensional data to identify static and dynamic objects. LIDAR data may be captured by a perception system for an autonomous vehicle and represented in a voxel space. Operations may include determining a drivable surface by parsing individual voxels to determine an orientation of a surface normal of a planar approximation of the voxelized data relative to a reference direction. Clustering techniques can be used to grow a ground plane including a plurality of locally flat voxels. Ground plane data can be set aside from the voxel space, and the remaining voxels can be clustered to determine objects. Voxel data can be analyzed over time to determine dynamic objects. Segmentation information associated with ground voxels, static object, and dynamic objects can be provided to a tracker and/or planner in conjunction with operating the autonomous vehicle.

Voxel based ground plane estimation and object segmentation

Growing community of inventors

Redwood City, CA, United States of America

Jesse Sol Levinson