Ambarella International LP

Giulio Bacchiani

A method of controlling actuators of a vehicle includes receiving pixel data corresponding to an area outside of the vehicle, receiving desired dynamic set-points for the vehicle from a path planning application of the vehicle, receiving sensor data from the vehicle, generating one or more real actuator commands for the vehicle in response to the sensor data from the vehicle, the desired dynamic set-points for the vehicle received from the path planning application of the vehicle, and one or more inferences made by a first trained neural network model using the sensor data from the vehicle and the desired dynamic set-points for the vehicle as input, and communicating the one or more real actuator commands for the vehicle to actuators of the vehicle. The one or more real actuator commands for the vehicle are generally presented at an output of the first trained neural network model. The first trained neural network model was generally trained by performing computer vision operations on the pixel data corresponding to the area outside the vehicle arranged as video frames to detect features in a first set of the video frames, determining a first set of measured dynamic set-points for the vehicle by applying visual odometry operations on the features detected in the first set of the video frames, and modifying a plurality of weights and bias values of an untrained neural network model based on the first set of measured dynamic set-points, the sensor data, and a training dataset comprising dynamic set-points and corresponding actuator values that are representative of one or more desired inferences.

Dynamic actuation map using neural network fed by visual odometry

An apparatus includes an interface and a processor. The interface may be configured to receive pixel data corresponding to an area outside of a vehicle and sensor data from said vehicle. The processor may be configured to generate one or more outputs in response to the sensor data based upon dynamic set-points for the vehicle and one or more inferences made by executing a first trained neural network model. The processor may be configured to process the pixel data arranged as video frames. The trained neural network model may have been trained by processing the pixel data arranged as video frames, performing computer vision operations to detect features in the video frames, determining the dynamic set-points for the vehicle by applying visual odometry operations on the features detected in the video frames, and modifying a plurality of weights of an untrained neural network model based on the dynamic set-points, the sensor data, and a training dataset comprising the dynamic set-points and corresponding actuator values that are representative of one or more desired inferences.

Growing community of inventors

Pesaro, Italy

Giulio Bacchiani