Systems and methods for training and evaluating long-context neural network based language models

Abstract

Embodiments described herein provide a method for configuring an artificial intelligence (AI) conversation bot to respond to a user query based on retrieved contextual documents. The method includes: receiving, via a communication interface, a user query comprising a natural language description of a topic; generating, by a first neural network based language model, one or more subtopics of the topic based on a first input prompt combining the topic and a first instruction to generate the one or more subtopics; generating, by the first neural network based language model, one or more statements for at least one of the subtopics based on a second input prompt combining the one or more subtopics and a second instruction to generate the one or more statements; and generating, by the first neural network based language model, at least one document containing a set of randomly selected statements from the one or more statements.