Gm Global Technolgoy Operations, Inc.

Embodied Intelligence Inc.

Ian Rust

Drew Gross

Solomon Bier

Kyle Vogt

Kevin Chu

Xi Chen

Yan Duan

Matthias Wisniowski

Andrew Amir Vaziri

Carlos Florensa

Nikhil Mishra

Yide Shentu

Mostafa Rohaninejad

Brooks Reed

Haoran Tang

Yung-Chang Ko

Matthew Graney

Yu Xuan Liu

Rita Ciarvino

Various embodiments of the present technology generally relate to robotic devices and artificial intelligence. More specifically, some embodiments relate to a robotic device for picking items from a bin and perturbing items in a bin. In some implementations, the device may include one or more computer-vision systems. A computer-vision system, in accordance with the present technology, may use at least two two-dimensional images to generate three-dimensional (3D) information about the bin and items in the bin. Based on the 3D information, a strategy for picking up items from the bin is determined. When no strategies with high probability of success exist, the robotic device may perturb the contents of the bin to create new available pick-up points and re-attempt to pick up an item.

Systems and methods for robotic picking

Various embodiments of the present technology generally relate to robotic devices and artificial intelligence. More specifically, some embodiments relate to a robotic device for picking items from a bin and perturbing items in a bin. The robotic device may include one or more picking elements and one or more perturbation elements for disturbing a present arrangement of items in the bin. In an exemplary embodiment, a perturbation element comprises a compressed air valve. In some implementations, the robotic device may also include one or more computer-vision systems. Based on image data from the one or more computer-vision systems, a strategy for picking up items from the bin is determined. When no strategies with high probability of success exist, the robotic device may perturb the contents of the bin to create new available pick-up points.

Systems and methods for robotic picking and perturbation

Systems and method are provided for controlling a vehicle. In one embodiment, a method includes calculating, via a router of a vehicle system that accesses road map data, at least one route to a destination based on the road map data, thereby producing route solution data. The vehicle system enters a remote assistance mode in response to remote assistance decision data received from a blockage arbiter of the vehicle system. In the remote assistance mode, the method includes determining, via the router, at least one road segment of the road map data that is permitted to be blacklisted, thereby producing permitted blacklist data. The method includes transmitting the permitted blacklist data and the route solution data, via a vehicle communications module of the vehicle system, to a remote vehicle assistance system. The method includes updating, via the router, the road map data to exclude at least one blacklisted road segment defined by the permitted blacklist data.

Vehicle remote assistance mode

In one embodiment, a method for controlling an actuator for a door of an autonomous vehicle comprises obtaining data pertaining to a current ride of an autonomous vehicle during operation of the autonomous vehicle; identifying, via a processor using the data, whether one or more circumstances are present that would require an adjustment of a baseline instruction for an automatic opening of the door by the autonomous vehicle via the actuator based on instructions provided to the actuator by the processor; determining an adjustment of the baseline instruction when one or more of the circumstances are present; receiving a request to open the door; and, upon receiving the request: providing the baseline instruction for the actuator to open the door, when none of the circumstances are present; and providing an alternate instruction for the actuator, based on the adjustment, when one or more of the circumstances are present.

Door actuator adjustment for autonomous vehicles

Systems and methods are provided for remotely assisting an autonomous vehicle. The method includes: aggregating sensor data from the autonomous vehicle; identifying an assistance-desired scenario; generating an assistance request based on the sensor data; transmitting the assistance request to a remote assistance interface; and receiving and processing a response to the assistance request. The remote assistance interface includes a remote assistance interface that is used in generating the response to the assistance request.

System and method for remotely assisting autonomous vehicle operation

Systems and method are provided for controlling an autonomous vehicle. In one embodiment, a method for controlling an autonomous vehicle comprises obtaining lidar data from one or more lidar sensors disposed on the autonomous vehicle during operation of the autonomous vehicle, generating a lidar point cloud using the lidar data, making an initial determination, via a processor onboard the autonomous vehicle, of a possible lidar point cloud anomaly based on a comparison of the lidar point cloud with prior lidar point cloud information stored in memory, receiving a notification from a remote module as to whether the possible lidar point cloud anomaly is a confirmed lidar point cloud anomaly, and taking one or more vehicle actions when there is a confirmed lidar point cloud anomaly.

Methods and systems for lidar point cloud anomalies

A method for controlling an autonomous vehicle using an external interface is disclosed. The method includes an external interface that is in operable communication with one or more control systems of the autonomous vehicle. The method allows an entity in proximity of an outside of the autonomous vehicle to interact with the external interface; receives from the entity one or more control inputs, and controls one or more operations of the autonomous vehicle based on the one or more control inputs.

System and method for externally interfacing with an autonomous vehicle

A system and method for providing routing instructions to one or more autonomous vehicles includes identifying a destination of an autonomous vehicle; identifying a starting position or an initial location of the autonomous vehicle; receiving autonomous vehicle sensor data; receiving one or more routing goals for a routing plan for the autonomous vehicle; generating one or more route modification parameters; and generating a route plan for the autonomous vehicle based on (a) the destination, (b) the starting position or the initial location, (c) the one or more route modification parameters and (d) the one or more routing goals.

System and method for autonomous vehicle fleet routing

Systems and method are provided for controlling an autonomous vehicle. The systems and methods obtain first and second time spaced point clouds based on three-dimensional position data, of a scene outside of the autonomous vehicle, from sensor of the autonomous vehicle. The systems and methods position align a static aspect of the scene in the first point cloud with a static aspect of the scene in the second point cloud to obtain position aligned first and second point clouds. The systems and methods determine, via a processor, a velocity of a moving object in the position aligned first and second point clouds. The systems and methods perform an autonomous vehicle control function using the velocity of the moving object.

Methods and systems for moving object velocity determination

Systems and method are provided for controlling an autonomous vehicle. In one embodiment, a method for controlling an autonomous vehicle comprises determining that assistance is required for continued movement of the autonomous vehicle; and facilitating movement of the autonomous vehicle via implementation, by a processor onboard the autonomous vehicle, of manual instructions provided from a remote user that is remote from the autonomous vehicle.

Expert mode for vehicles

An autonomous vehicle retrofit system includes: a central computer; and a braking interface to decelerate a wheel of a vehicle via actuation of a brake caliper, the braking interface further including a brake pedal; a first master cylinder assembly, mechanically coupled to the brake pedal such that actuation of the brake pedal causes actuation of the first master cylinder assembly; a second master cylinder assembly, coupled to the central computer such that the central computer controls actuation of the second master cylinder assembly; and an actuator selector, hydraulically coupled to the brake caliper, that selectively actuates the brake caliper in response to actuation of at least one of the first master cylinder assembly and the second master cylinder assembly.

Growing community of inventors

San Francisco, CA, United States of America

Ian Rust