Method and system of context window engineering for large language models fine-tuned for conversations

Abstract

Methods and systems for context window management of large language models are disclosed. An example method includes: receiving a message in a user language; preprocessing the message; setting a static context based on a topic of the message for a text sequence including the message, if the topic is new or different from a topic of a preceding message; attaching the static context to a context window; attaching a long term dynamic context to the context window based on the message and one or more previous messages in the text sequence; attaching a short term dynamic context to the context window based on the message; providing the context window to a language model server; receiving a database access command based on the context window from the language model server; providing the database access command to a database; and receiving a result response to the database access command from the database.