Extracting features from sensor data

Abstract

An encoder is trained together with a perception component based on a training set comprising unannotated sensor data sets and annotated sensor data sets in a sequence of multiple training steps. Each training step comprises: in a first phase of the training step, updating the set of encoder parameters based on the unannotated sensor data sets, with the aim of optimizing a self-supervised loss function, without updating the set of task-specific parameters of the perception component, and in a second phase of the training step, updating the set of task-specific parameters based on the annotated sensor data sets, with the aim of optimizing a task-specific loss function, wherein the encoder as updated in the first phase of that training step processes a data representation of each annotated sensor data set to extract features therefrom, wherein the perception component processes the extracted features to compute an output therefrom, and wherein the task-specific loss is defined on the output and the associated annotation for each annotated sensor data set for learning a desired perception task. In performing the sequence of multiple training steps, the method alternates repeatedly between the first phase and the second phase.