Adobe, Inc.

Robert Boschgmbh

Accenture Global Services Gmbh

Roku, Inc.

Doo Soon Kim

Trung Huu Bui

Franck Dernoncourt

Walter Chang

Kunal Verma

Peter Z Yeh

Quan Anh Tran

Lidan Wang

Tuan Manh Lai

Praful Chandra Mangalath

Jonathan Worthen Brandt

Fuliang Weng

Zhe Feng

Chen Fang

Aliakbar Darabi

Colin Anil Puri

Nedim Lipka

Rajiv Jain

Lin Zhao

Carl Dockhorn

Varun Manjunatha

Seokhwan Kim

Amir Pouran Ben Veyseh

Moumita Sinha

Seunghyun Yoon

Yinglan Ma

Seung-hyun Yoon

Bao Nguyen

Yu Gong

Rakesh Ravuru

Chen Chen

Minsuk Heo

Ramesh Radhakrishna Manuvinakurike

Oliver Markus Michael Brdiczka

Anthony M Colas

Logan Lebanoff

Sean MacAvaney

Meryem M'hamdi

Arman Cohan

Hyounghun Kim

Ripul Bhutani

Behnam Asefisaray

Zei-Chan Yeh

Jacqueline Brixey

Tzu-Hsiang Lin

Systems and methods for natural language processing are described. Embodiments of the present disclosure identify a task set including a plurality of pseudo tasks, wherein each of the plurality of pseudo tasks includes a support set corresponding to a first natural language processing (NLP) task and a query set corresponding to a second NLP task; update a machine learning model in an inner loop based on the support set; update the machine learning model in an outer loop based on the query set; and perform the second NLP task using the machine learning model.

Cross-lingual meta-transfer learning adaptation to natural language understanding

Some embodiments include a transcription knowledge graph that can resolve automatic speech recognition (ASR) engine output errors. In some embodiments, a transcription knowledge graph can utilize data from past sessions of the ASR engine to form a voice graph that can be analyzed to determine a correlation between a mis-transcription (error text) and the correct transcription (correct text). Thus, ASR engine outputs, even if they include a mis-transcription, can be adjusted to the correct transcription. Further, the correct transcriptions and the voice graph can be used to train machine learning (ML) algorithms to generate numerical representations of an entity. The ML algorithms can be applied to a transcription to correctly identify a corresponding entity label, even if the transcription was not utilized in the voice graph to train the ML algorithm.

Transcription knowledge graph

Disclosed herein are system, apparatus, article of manufacture, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for tailoring and censoring content based on audience detected. An example embodiment operates by detecting an audience within a vicinity of a media device based on identifying information received by the media device, determining a category of the audience with a user identification system based on the identifying information, identifying a content tailoring rule for the audience based on the category of the audience, retrieving a content to be played by the media device, and modifying the content based on the content tailoring rule and a category label of the content.

Tailoring and censoring content based on a detected audience

Systems and methods for coreference resolution are provided. One aspect of the systems and methods includes inserting a speaker tag into a transcript, wherein the speaker tag indicates that a name in the transcript corresponds to a speaker of a portion of the transcript; encoding a plurality of candidate spans from the transcript based at least in part on the speaker tag to obtain a plurality of span vectors; extracting a plurality of entity mentions from the transcript based on the plurality of span vectors, wherein each of the plurality of entity mentions corresponds to one of the plurality of candidate spans; and generating coreference information for the transcript based on the plurality of entity mentions, wherein the coreference information indicates that a pair of candidate spans of the plurality of candidate spans corresponds to a pair of entity mentions that refer to a same entity.

Systems and methods for coreference resolution

Embodiments of the disclosure provide a machine learning model for generating a predicted executable command for an image. The learning model includes an interface configured to obtain an utterance indicating a request associated with the image, an utterance sub-model, a visual sub-model, an attention network, and a selection gate. The machine learning model generates a segment of the predicted executable command from weighted probabilities of each candidate token in a predetermined vocabulary determined based on the visual features, the concept features, current command features, and the utterance features extracted from the utterance or the image.

Systems and methods for image processing using natural language

Systems and methods for natural language processing are described. One or more embodiments of the present disclosure receive a document comprising a plurality of words organized into a plurality of sentences, the words comprising an event trigger word and an argument candidate word, generate word representation vectors for the words, generate a plurality of document structures including a semantic structure for the document based on the word representation vectors, a syntax structure representing dependency relationships between the words, and a discourse structure representing discourse information of the document based on the plurality of sentences, generate a relationship representation vector based on the document structures, and predict a relationship between the event trigger word and the argument candidate word based on the relationship representation vector.

Inducing rich interaction structures between words for document-level event argument extraction

The present disclosure relates to systems, methods, and non-transitory computer readable media for generating coachmarks and concise instructions based on operation descriptions for performing application operations. For example, the disclosed systems can utilize a multi-task summarization neural network to analyze an operation description and generate a coachmark and a concise instruction corresponding to the operation description. In addition, the disclosed systems can provide a coachmark and a concise instruction for display within a user interface to, directly within a client application, guide a user to perform an operation by interacting with a particular user interface element.

Generating in-app guided edits including concise instructions and coachmarks

This disclosure relates to methods, non-transitory computer readable media, and systems that can classify term sequences within a source text based on textual features analyzed by both an implicit-class-recognition model and an explicit-class-recognition model. For example, by applying machine-learning models for both implicit and explicit class recognition, the disclosed systems can determine a class corresponding to a particular term sequence within a source text and identify the particular term sequence reflecting the class. The dual-model architecture can equip the disclosed systems to apply (i) the implicit-class-recognition model to recognize implicit references to a class in source texts and (ii) the explicit-class-recognition model to recognize explicit references to the same class in source texts.

Classifying terms from source texts using implicit and explicit class-recognition-machine-learning models

Systems and methods for sentence fusion are described. Embodiments receive coreference information for a first sentence and a second sentence, wherein the coreference information identifies entities associated with both a term of the first sentence and a term of the second sentence, apply an entity constraint to an attention head of a sentence fusion network, wherein the entity constraint limits attention weights of the attention head to terms that correspond to a same entity of the coreference information, and predict a fused sentence using the sentence fusion network based on the entity constraint, wherein the fused sentence combines information from the first sentence and the second sentence.

Learning to fuse sentences with transformers for summarization

The present disclosure relates to systems, methods, and non-transitory computer-readable media for generating pairs of natural language queries and corresponding query-language representations. For example, the disclosed systems can generate a contextual representation of a prior-generated dialogue sequence to compare with logical-form rules. In some implementations, the logical-form rules comprise trigger conditions and corresponding logical-form actions for constructing a logical-form representation of a subsequent dialogue sequence. Based on the comparison to logical-form rules indicating satisfaction of one or more trigger conditions, the disclosed systems can perform logical-form actions to generate a logical-form representation of a subsequent dialogue sequence. In turn, the disclosed systems can apply a natural-language-to-query-language (NL2QL) template to the logical-form representation to generate a natural language query and a corresponding query-language representation for the subsequent dialogue sequence.

Utilizing logical-form dialogue generation for multi-turn construction of paired natural language queries and query-language representations

This disclosure describes one or more implementations of a text sequence labeling system that accurately and efficiently utilize a joint-learning self-distillation approach to improve text sequence labeling machine-learning models. For example, in various implementations, the text sequence labeling system trains a text sequence labeling machine-learning teacher model to generate text sequence labels. The text sequence labeling system then creates and trains a text sequence labeling machine-learning student model utilizing the training and the output of the teacher model. Upon the student model achieving improved results over the teacher model, the text sequence labeling system re-initializes the teacher model with the learned model parameters of the student model and repeats the above joint-learning self-distillation framework. The text sequence labeling system then utilizes a trained text sequence labeling model to generate text sequence labels from input documents.

Utilizing a joint-learning self-distillation framework for improving text sequential labeling machine-learning models

An activity planning system comprises a knowledge base, a query processor, and a temporal reasoner. A query including temporal constraints is input into the query processor. The query processor converts the query into a formal representation. The formal representation is a formal graphical semantic representation grounded on an ontology defined in the knowledge base. The temporal reasoner processes the query representation output by the query processor against the knowledge base which defines a set of object. For each object, the temporal reasoner produces a normalized score from 0 to 1 to indicate the degree of how likely the object satisfies the temporal constraints imposed by the query.

Active activity planning system and method for supporting temporal constraints

Dialog system training techniques using a simulated user system are described. In one example, a simulated user system supports multiple agents. The dialog system, for instance, may be configured for use with an application (e.g., digital image editing application). The simulated user system may therefore simulate user actions involving both the application and the dialog system which may be used to train the dialog system. Additionally, the simulated user system is not limited to simulation of user interactions by a single input mode (e.g., natural language inputs), but also supports multimodal inputs. Further, the simulated user system may also support use of multiple goals within a single dialog session

Dialog system training using a simulated user system

In implementations of systems for generating suggestions with knowledge graph embedding vectors, a computing device implements a suggestion system to receive input data describing user interactions with an application for editing digital content. The suggestion system generates input embedding vectors based on the user interactions with the application and determines an item based on the input embedding vectors and knowledge graph embedding vectors generated from nodes of a knowledge graph describing a tutorial for editing digital content. The suggestion system generates an indication of the item for display in a user interface of a display device.

Systems for generating suggestions with knowledge graph embedding vectors

The present disclosure relates to utilizing a graph neural network to accurately and flexibly identify text phrases that are relevant for responding to a query. For example, the disclosed systems can generate a graph topology having a plurality of nodes that correspond to a plurality of text phrases and a query. The disclosed systems can then utilize a graph neural network to analyze the graph topology, iteratively propagating and updating node representations corresponding to the plurality of nodes, in order to identify text phrases that can be used to respond to the query. In some embodiments, the disclosed systems can then generate a digital response to the query based on the identified text phrases.

Utilizing a graph neural network to identify supporting text phrases and generate digital query responses

Growing community of inventors

San Jose, CA, United States of America

Doo Soon Kim