Adobe, Inc.

Hanieh Deilamsalehy

Franck Dernoncourt

Aseem Agarwala

Lubomira A Dontcheva

Joel Richard Brandt

Hijung Shin

Dingzeyu Li

Joy Oakyung Kim

Kim Pascal Pimmel

Cristin Ailidh Fraser

Haoran Cai

Sajad Sotudeh Gharebagh

Anvesh Rao Vijjini

The present disclosure relates to systems, non-transitory computer-readable media, and methods for training and utilizing a transcript segmentation neural network to segment audio recording transcripts. In particular, in one or more embodiments, the disclosed systems implement a pretraining technique for a transcript segmentation neural network based on a specialized dataset that includes contextual information about stored sentences or conversations. For example, the discloses system train a transcript segmentation neural network based on contextual data that indicates semantic similarities and/or distances between sentences of a digital document. In some cases, the disclosed systems also (or alternatively) train a transcript segmentation neural network based on curricular data generated by a classification model. Moreover, in some cases, the disclosed system use the trained transcript segmentation neural network to generate a segmented audio transcript.

Curricular next conversation prediction pretraining for transcript segmentation

Embodiments of the present invention provide systems, methods, and computer storage media for a visual and text search interface used to navigate a video transcript. In an example embodiment, a freeform text query triggers a visual search for frames of a loaded video that match the freeform text query (e.g., frame embeddings that match a corresponding embedding of the freeform query), and triggers a text search for matching words from a corresponding transcript or from tags of detected features from the loaded video. Visual search results are displayed (e.g., in a row of tiles that can be scrolled to the left and right), and textual search results are displayed (e.g., in a row of tiles that can be scrolled up and down). Selecting (e.g., clicking or tapping on) a search result tile navigates a transcript interface to a corresponding portion of the transcript.

Visual and text search interface for text-based video editing

Embodiments of the present invention provide systems, methods, and computer storage media for segmenting a transcript into paragraphs. In an example embodiment, a transcript is segmented to start a new paragraph whenever there is a change in speaker and/or a long pause in speech. If any remaining paragraphs are longer than a designated length or duration (e.g., 50 or 100 words), each of those paragraphs is segmented using dynamic programming to minimize a cost function that penalizes candidate paragraphs based on divergence from a target paragraph length and/or that rewards candidate paragraphs that group semantically similar sentences. As such, the transcript is visualized, segmented at the identified paragraphs.

Transcript paragraph segmentation and visualization of transcript paragraphs

Techniques for training for and performing abstractive text summarization are disclosed. Such techniques include, in some embodiments, obtaining textual content, and generating a reconstruction of the textual content using a trained language model, the reconstructed textual content comprising an abstractive summary of the textual content generated based on relative importance parameters associated with respective portions of the textual content. In some cases, the trained language model includes a neural network language model that has been trained by identifying a plurality of discrete portions of training textual content, receiving the plurality of discrete portions of the training textual content as input to the language model, and predicting relative importance parameters associated with respective ones of the plurality of discrete portions of the training textual content, the relative importance parameters each being based at least on one or more linguistic similarity measures with respect to a ground truth.

Training a model for performing abstractive text summarization

Growing community of inventors

Seattle, WA, United States of America

Hanieh Deilamsalehy