Location and navigation assistant

Abstract

The techniques presented herein are directed to a system for automated navigation assistance in indoor environments. In general, navigating an unfamiliar indoor environment can be a daunting task. This is especially true for large environments such as corporate offices, university campuses, shopping centers, and hospitals. Accordingly, the present system utilizes a reference database containing information associated with the indoor environment such as images depicting various locations, annotations describing the images, and floorplans illustrating the layout of the indoor environment. The reference database is deployed in conjunction with a multimodal language model that utilizes the collected knowledge of the reference database to provide location identification and intuitive navigation instructions. Often referred to as retrieval-augmented generation (RAG), the reference database enables the multimodal language model to respond to user requests using domain-specific information and ensure accurate outputs that are relevant to an end user's context (e.g., the indoor environment).