Identification and analysis of an environment using image-based large language model processing

Abstract

A system captures sets of images of a physical building. Each set of images corresponds to a capture time and a location within the physical building. The system applies the sets of images to a large language model, which is configured to generate a description of an image and a description of changes between the image and a previous image. The previous image may be captured closest in time before the image and correspond to a same location as the image. The system stores the generated descriptions in a database. The system receives a query associated with a target time and a target location within the physical building, and accesses the target description and the target description of changes associated with the image and the previous image. The system generates a query response based at least in part on the target description and target description of changes.