Curricular next conversation prediction pretraining for transcript segmentation

Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for training and utilizing a transcript segmentation neural network to segment audio recording transcripts. In particular, in one or more embodiments, the disclosed systems implement a pretraining technique for a transcript segmentation neural network based on a specialized dataset that includes contextual information about stored sentences or conversations. For example, the discloses system train a transcript segmentation neural network based on contextual data that indicates semantic similarities and/or distances between sentences of a digital document. In some cases, the disclosed systems also (or alternatively) train a transcript segmentation neural network based on curricular data generated by a classification model. Moreover, in some cases, the disclosed system use the trained transcript segmentation neural network to generate a segmented audio transcript.