Kpmg Llp

Charles Scott McAllister

Viral Chawda

John Henry Rudisill

Arun Aggarwal

Kimball Hill

An embodiment of the present invention is directed to a multimodal assistant for mechanics and technicians in military warehouses using large language models. An exemplary system provides a conversational assistant to mechanics and technicians working in challenging environments, such as shop or warehouse floors with heavy industrial equipment and componentry. The innovative system may utilize multimodal large language models as well as image segmentation techniques to efficiently retrieve relevant information from asset reference materials, documentation and other sources of instructional information. The system may also use multimodal large language models to extract information from the retrieved documents and/or other data sources to provide guidance on performing discrete tasks related to the asset, such as routine maintenance, part replacement, services and/or other related actions.

System and method for implementing a multimodal assistant using large language models

An embodiment of the present invention is directed to a combination of two deep-learning computer vision models—customized with post-processing—wrapped in a mobile application that is backed by an Application Programming Interface (API) supporting concurrent mobile users to accomplish asset serialization tasks in a warehouse or other storage environment.

Growing community of inventors

Silver Spring, MD, United States of America

Charles Scott McAllister