Adobe, Inc.

Carl Dockhorn

Franck Dernoncourt

Trung Huu Bui

Sean Fitzgerald

Yu Gong

Sachin Soni

Walter Wei-Tuh Chang

Frederic Thevenet

Laurie Marie Byrum

Sasha Spala

Ragunandan Rao Malangully

Luke A Rymarz

Ajinkya Gorakhnath Kale

Nicholas Miller

Jason Guthrie Waters

Tushar Dublish

Joon Kim

Walter Chang

Ajay Jain

Abhishek Jain

Doo Soon Kim

Seokhwan Kim

Lidan Wang

Seung-hyun Yoon

Sebastian Gehrmann

Shagun Kush

Nikita Kapoor

David W McAllister

Kishore Aradhya

Vikas Vaishnav

Ashish Rawat

Techniques and systems are described for performing semantic text searches. A semantic text-searching solution uses a machine learning system (such as a deep learning system) to determine associations between the semantic meanings of words. These associations are not limited by the spelling, syntax, grammar, or even definition of words. Instead, the associations can be based on the context in which characters, words, and/or phrases are used in relation to one another. In response to detecting a request to locate text within an electronic document associated with a keyword, the semantic text-searching solution can return strings within the document that have matching and/or related semantic meanings or contexts, in addition to exact matches (e.g., string matches) within the document. The semantic text-searching solution can then output an indication of the matching strings.

Model-based semantic text searching

Techniques are provided herein for generating improved document summaries that consider the amount of time that has passed since the user last accessed the document. The length of time that has passed since the user has accessed each previous portion of the document is used as a variable to determine how much the summary should focus on each of the previously read sections of the document. When a document is accessed by a user, a relevance score is assigned to content from previously accessed sections of that document, where the relevance score is weighted based on how long ago each of the sections was accessed by the user. Once the various content items of previous sections have been provided relevance scores, selected sentences with the highest relevance scores are fed to a deep learning sequence-to-sequence model is used to build the document summary.

Generating time-based recaps of documents using a deep learning sequence to sequence model

Systems, methods, and non-transitory computer-readable media are disclosed that collect and analyze annotation performance data to generate digital annotations for evaluating and training automatic electronic document annotation models. In particular, in one or more embodiments, the disclosed systems provide electronic documents to annotators based on annotator topic preferences. The disclosed systems then identify digital annotations and annotation performance data such as a time period spent by an annotator in generating digital annotations and annotator responses to digital annotation questions. Furthermore, in one or more embodiments, the disclosed systems utilize the identified digital annotations and the annotation performance data to generate a final set of reliable digital annotations. Additionally, in one or more embodiments, the disclosed systems provide the final set of digital annotations for utilization in training a machine learning model to generate annotations for electronic documents.

Generating digital annotations for evaluating and training automatic electronic document annotation models

The disclosure describes one or more embodiments of a structured text summary system that generates structured text summaries of digital documents based on an interactive graphical user interface. For example, the structured text summary system can collaborate with users to create structured text summaries of a digital document based on automatically generating document tags corresponding to the digital document, determining segments of the digital document that correspond to a selected document tag, and generating structured text summaries for those document segments.

Generating structured text summaries of digital documents using interactive collaboration

Embodiments of the present invention provide systems, methods, and computer storage media for techniques for identifying textual similarity and performing answer selection. A textual-similarity computing model can use a pre-trained language model to generate vector representations of a question and a candidate answer from a target corpus. The target corpus can be clustered into latent topics (or other latent groupings), and probabilities of a question or candidate answer being in each of the latent topics can be calculated and condensed (e.g., downsampled) to improve performance and focus on the most relevant topics. The condensed probabilities can be aggregated and combined with a downstream vector representation of the question (or answer) so the model can use focused topical and other categorical information as auxiliary information in a similarity computation. In training, transfer learning may be applied from a large-scale corpus, and the conventional list-wise approach can be replaced with point-wise learning.

Answer selection using a compare-aggregate model with language model and condensed similarity information from latent clustering

Techniques are disclosed for generating a structured transcription from a speech file. In an example embodiment, a structured transcription system receives a speech file comprising speech from one or more people and generates a navigable structured transcription object. The navigable structured transcription object may comprise one or more data structures representing multimedia content with which a user may navigate and interact via a user interface. Text and/or speech relating to the speech file can be selectively presented to the user (e.g., the text can be presented via a display, and the speech can be aurally presented via a speaker).

Emphasizing key points in a speech file and structuring an associated transcription

Highlighting key portions of text within a document is described. A document having text is obtained, and key portions of the document are determined using summarization techniques. Key portion data indicative of the key portions is generated and maintained for output to generate a highlighted document in which highlight overlays are displayed over or proximate the determined key portions of the text within the document. In one or more implementations, reader interactions with the highlighted document are monitored to generate reader feedback data. The reader feedback data may then be combined with the output of the summarization techniques in order to adjust the determined key portions. In some cases, the reader feedback data may also be used to improve the summarization techniques.

Highlighting key portions of text within a document

In embodiments of photo chapters organization, a photo analyzer is implemented to receive a set of digital photos, and generate a photo sequence of the digital photos based on a respective timestamp of each photo. The photo sequence includes a first photo based on an earliest timestamp and a last photo based on a latest timestamp. The photo analyzer creates a photo chapter that includes the first photo. To organize the digital photos into photo chapters, the photo analyzer can determine whether a next photo in the photo sequence is to be added to a current photo chapter, and then either adds the next photo to the current photo chapter, or creates a next photo chapter that includes the next photo. The photo analyzer can repeat to organize each next photo in the photo sequence into the photo chapters until the last photo is included in a last photo chapter.

Photo chapters organization

In embodiments of photo importance determination, a photo analyzer is implemented to analyze the image content of each photo in a set of digital photos, and determine similar photos based on the image content and metadata of the digital photos. The photo analyzer can then create stacks of the similar photos and determine a representative photo from the similar photos in each stack. The photo analyzer can then determine a display sequence to display non-stacked photos and the representative photos of each stack. The photo analyzer can also receive viewer feedback associated with the digital photos being displayed for viewing, and then determine a different representative photo from the similar photos in each of the stacks based on the viewer feedback. The photo analyzer can also determine a revised display sequence of the non-stacked photos and the representative photos of the stacks based on the viewer feedback.

Photo importance determination

Growing community of inventors

San Jose, CA, United States of America

Carl Dockhorn