Decoder architecture for early-stop automated retrieval augmented generation

Abstract

Aspects of the present disclosure relate to a decoder model architecture that is optimized for retrieval augmented generation (RAG) applications. Embodiments include a generative machine learning model that comprises one or more hidden decoder layers and an output layer. In certain embodiments, the output layer includes a stop head layer configured to generate, based on a hidden state output generated by a given hidden layer of the one or more hidden decoder layers, a stop output indicating a likelihood of performing a subsequent context retrieval. Some embodiments provide that the output layer includes a softmax head layer parallel to the stop head layer. In certain embodiments, each of the one or more hidden decoder layers and the output layer comprise a plurality of neurons organized in an array, wherein each neuron comprises a register, a microprocessor, and at least one input. The neurons may be connected using synaptic circuitry.