At&t Intellectual Property I, L.p.

At+t Corp.

Nuance Communications, Inc.

Interactions Corporation

Hyundai Motor Company

University of Texas System

Kia Corporation

Cerence Operating Company

Srinivas Bangalore

Junlan Feng

Mazin E Gilbert

Michael J Johnston

Mazin G Rahim

Narendra Kumar Gupta

Patrick Guy Haffner

Taniya Mishra

David Crawford Gibbon

Behzad Shahraray

Zhu Liu

Giuseppe Riccardi

Amanda Joy Stent

Vivek Kumar Rangarajan Sridhar

John Chen

Jay Gordon Wilpon

Luciano De Andrade Barbosa

Giuseppe Di Fabbrizio

Dan Melamed

Ann K Syrdal

Juergen Schroeter

Robert Bell

David Eugene Schulz

Charles Douglas Blewett

Sumit Chopra

Diamantino Antonio Caseiro

Ilija Zeljkovic

Owen Christopher Rambow

Svetlana Stoyanchev

Stephan Kanthak

Hyuckchul Jung

Michael James Robert Johnston

Kateryna Kuksenok

Dilek Zeynep Hakkani-Tur

Alistair D Conkie

Suhrid Balakrishnan

Piotr Mirowski

David J Smith

Emil Ettelaie

Mercan Topkara

Umut Topkara

Ryan Price

Mikhail Mike Atallah

Brian David Lester

Mahnoosh Mehrabani

Nicholas Ruiz

A task-oriented dialog system determines an endpoint in a user utterance by receiving incremental portions of a user utterance that is provided in real time during a task-oriented communication session between a user and a virtual agent (VA). The task-oriented dialog system recognizes words in the incremental portions using an automated speech recognition (ASR) model and generates semantic information for the incremental portions of the utterance by applying a natural language processing (NLP) model to the recognized words. An acoustic-prosodic signature of the incremental portions of the utterance is generated using an acoustic-prosodic model. The task-oriented dialog system can generate a feature vector that represents the incrementally recognized words, the semantic information, the acoustic-prosodic signature, and corresponding confidence scores of the model outputs. A model is applied to the feature vector to identify a likely endpoint in the user utterance.

Utterance endpointing in task-oriented conversational systems

A natural language understanding (NLU) system generates in-place annotations for natural language utterances or other types of time-based media based on stand-off annotations. The in-place annotations are associated with particular sub-sequences of an annotation, which provides richer information than stand-off annotations, which are associated only with an utterance as a whole. To generate the in-place annotations for an utterance, the NLU system applies an encoder network and a decoder network to obtain attention weights for the various tokens within the utterance. The NLU system disqualifies tokens of the utterance based on their corresponding attention weights, and selects highest-scoring contiguous sequences of tokens between the disqualified tokens. In-place annotations are associated with the selected sequences.

Annotating and modeling natural language semantics through annotation conversion

In an embodiment of a messaging system, a method for presenting a commercial message to a user is provided. A target language in which the user is comfortable communicating may be determined based on at least one communication received by the user or at least one communication provided by the user. The commercial message may be presented to the user in the target language.

Automatic translation of advertisements

A system, method and computer-readable storage devices are for processing natural language commands, such as commands to a robotic arm, using a Tag & Parse approach to semantic parsing. The system first assigns semantic tags to each word in a sentence and then parses the tag sequence into a semantic tree. The system can use statistical approach for tagging, parsing, and reference resolution. Each stage can produce multiple hypotheses, which are re-ranked using spatial validation. Then the system selects a most likely hypothesis after spatial validation, and generates or outputs a command. In the case of a robotic arm, the command is output in Robot Control Language (RCL).

System and method for semantic processing of natural language commands

Systems and methods are disclosed herein for discerning aspects of user speech to determine user intent and/or other acoustic features of a sound input without the use of an ASR engine. To this end, a processor may receive a sound signal comprising raw acoustic data from a client device, and divides the data into acoustic units. The processor feeds the acoustic units through a first machine learning model to obtain a first output and determines a first mapping, using the first output, of each respective acoustic unit to a plurality of candidate representations of the respective acoustic unit. The processor feeds each candidate representation of the plurality through a second machine learning model to obtain a second output, determines a second mapping, using the second output, of each candidate representation to a known condition, and determines a label for the sound signal based on the second mapping.

Extracting natural language semantics from speech without the use of speech recognition

Disclosed herein are systems, computer-implemented methods, and computer-readable media for dialog modeling. The method includes receiving spoken dialogs annotated to indicate dialog acts and task/subtask information, parsing the spoken dialogs with a hierarchical, parse-based dialog model which operates incrementally from left to right and which only analyzes a preceding dialog context to generate parsed spoken dialogs, and constructing a functional task structure of the parsed spoken dialogs. The method can further either interpret user utterances with the functional task structure of the parsed spoken dialogs or plan system responses to user utterances with the functional task structure of the parsed spoken dialogs. The parse-based dialog model can be a shift-reduce model, a start-complete model, or a connection path model.

System and method for dialog modeling

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for collecting web data in order to create diverse language models. A system configured to practice the method first crawls, such as via a crawler operating on a computing device, a set of documents in a network of interconnected devices according to a visitation policy, wherein the visitation policy is configured to focus on novelty regions for a current language model built from previous crawling cycles by crawling documents whose vocabulary considered likely to fill gaps in the current language model. A language model from a previous cycle can be used to guide the creation of a language model in the following cycle. The novelty regions can include documents with high perplexity values over the current language model.

System and method for building diverse language models

A system and method are disclosed for generating customized text-to-speech voices for a particular application. The method comprises generating a custom text-to-speech voice by selecting a voice for generating a custom text-to-speech voice associated with a domain, collecting text data associated with the domain from a pre-existing text data source and using the collected text data, generating an in-domain inventory of synthesis speech units by selecting speech units appropriate to the domain via a search of a pre-existing inventory of synthesis speech units, or by recording the minimal inventory for a selected level of synthesis quality. The text-to-speech custom voice for the domain is generated utilizing the in-domain inventory of synthesis speech units. Active learning techniques may also be employed to identify problem phrases wherein only a few minutes of recorded data is necessary to deliver a high quality TTS custom voice.

System and method for generating customized text-to-speech voices

Machine translation is used to leverage the semantic properties (e.g., intent) already known for one natural language for use in another natural language. In a first embodiment, the corpus of a first language is translated to each other language of interest using machine translation, and the corresponding semantic properties are transferred to the translated corpuses. Semantic models can then be generated from the translated corpuses and the transferred semantic properties. In a second embodiment, given a first language for which there is a semantic model, if a query is received in a second, different language lacking its own semantic model, machine translation is used to translate the query into the first language. Then, the semantic model for the first language is applied to the translated query, thereby obtaining the semantic properties for the query, even though no semantic model existed for the language in which the query was specified.

Bootstrapping multilingual natural language understanding via machine translation

Extracting, from user activity data, quantitative attributes and qualitative attributes collected for users having user profiles. The quantitative attributes and the qualitative attributes are extracted during a specified time period determined before the user activity data is collected. Values for the quantitative attributes and the qualitative attributes are plotted, and subsets of the user profiles are clustered into separate group of users based on the plotted values. Delivering a product related content to the groups of users based on the clustering.

User profile and its location in a clustered profile landscape

A natural language processing system has a hierarchy of user intents related to a domain of interest, the hierarchy having specific intents corresponding to leaf nodes of the hierarchy, and more general intents corresponding to ancestor nodes of the leaf nodes. The system also has a trained understanding model that can classify natural language utterances according to user intent. When the understanding model cannot determine with sufficient confidence that a natural language utterance corresponds to one of the specific intents, the natural language processing system traverses the hierarchy of intents to find a more general user intent that is related to the most applicable specific intent of the utterance and for which there is sufficient confidence. The general intent can then be used to prompt the user with questions applicable to the general intent to obtain the missing information needed for a specific intent.

Underspecification of intents in a natural language processing system

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for generating domain-specific speech recognition models for a domain of interest by combining and tuning existing speech recognition models when a speech recognizer does not have access to a speech recognition model for that domain of interest and when available domain-specific data is below a minimum desired threshold to create a new domain-specific speech recognition model. A system configured to practice the method identifies a speech recognition domain and combines a set of speech recognition models, each speech recognition model of the set of speech recognition models being from a respective speech recognition domain. The system receives an amount of data specific to the speech recognition domain, wherein the amount of data is less than a minimum threshold to create a new domain-specific model, and tunes the combined speech recognition model for the speech recognition domain based on the data.

System and method for rapid customization of speech recognition models

Systems, methods, and computer-readable storage devices for receiving an utterance from a user and analyzing the utterance to identify the demographics of the user. The system then analyzes the utterance to determine the prosody of the utterance, and retrieves from the Internet data associated with the determined demographics. Using the retrieved data, the system retrieves, also from the Internet, recorded speech matching the identified prosody. The recorded speech, which is based on the demographic data of the utterance and has a prosody matching the utterance, is then saved to a database for future use in generating speech specific to the user.

System and method for creating voice profiles for specific demographics

Growing community of inventors

Morristown, NJ, United States of America

Srinivas Bangalore