Feature detection using language models

Abstract

In various examples, feature identification using language models for autonomous and semi-autonomous systems and applications is described herein. Systems and methods described herein may use a language model(s) to determine information associated with features, such as surface markings, within an environment. For example, sensor data may be used to generate one or more images or other sensor data representations corresponding to an environment. The image(s) may then be processed to generate input data (e.g., input tokens) that is applied to the language model(s). Based at least on processing the input data, the language model(s) may be trained to output data (e.g., output tokens) representing information associated with one or more features. Additionally, the output data may be used to determine the information associated the feature(s) within the environment, where the information may then be used to update a map and/or navigate one or more machines within the environment.