Determining topic chapters for digital videos utilizing a sliding window and video segmentation machine learning models

Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for segmenting digital videos into topic chapters utilizing a sliding window and video segmentation models. Specifically, the disclosed systems utilize a sliding window to divide a digital video into overlapping segments, each segment including a subset of sentences of a transcript of the video and corresponding video frames for a given time window of the digital video. Further, the disclosed systems generate, for each overlapping segment, topic-boundary label predictions for the subset of sentences. Specifically, the disclosed systems generate text representations for the sentences using a text encoder and frame representations for the corresponding video frames using a frame encoder. Moreover, the disclosed systems generate the topic-boundary label predictions based on the text representations and the frame representations. Additionally, the disclosed systems generate a topic-boundary label for each sentence of the transcript based on the topic-boundary label predictions.