Microsoft Technology Licensing, LLC

Nuance Communications, Inc.

Rev.com, Inc.

Jean-Philippe Robichaud

Ruhi Sarikaya

Paul Anthony Crook

Matthieu Hebert

Omar Zia Khan

Marius Alexandru Marin

Alexandre Rochette

Vipul Agarwal

Christopher M Parisien

Migüel Jetté

Nikhil Holenarsipur Ramesh

Alexei Skurikhin

Petrov Evgeny Stanislavovich

Nicolae Duta

Daniel Boies

Jerome Tremblay

Vasiliy Radostev

Amjad Almahairi

Lakshmish Kaushik

Maryse Boisvert

Joshua Ian Dong

Quinten McNamara

Nishchal Bhandari

Michelle Kai Yu Huang

In one embodiment, a method for transcript generation includes receiving an audio file and dividing it into a plurality of chunks. The method further includes sending each instance of the plurality of chunks to a speech service module. The method further includes converting speech to text for each instance of the plurality of chunks and returning the text for each instance of the plurality of chunks. The method further includes merging the text for each instance of the plurality of chunks to yield an audio file transcript and sending the audio file and chunks to a diarization module. The method further includes performing first pass diarization on the chunks to yield a plurality of diarized chunks and performing second pass diarization on the plurality of diarized chunks and the audio file to yield a diarized audio file. The method further includes merging the files to yield a final transcript.

Systems and methods for a two pass diarization, automatic speech recognition, and transcript generation

A method of determining an alignment sequence between a reference sequence of symbols and a hypothesis sequence of symbols includes loading a reference sequence of symbols to a computing system and creating a reference finite state automaton for the reference sequence of symbols. The method further includes loading a hypothesis sequence of symbols to the computing system and creating a hypothesis finite state automaton for the hypothesis sequence of symbols. The method further includes traversing the reference finite state automaton, adding new reference arcs and new reference transforming properties arcs and traversing the hypothesis finite state automaton, adding new hypothesis arcs and new hypothesis transforming properties arcs. The method further includes composing the hypothesis finite state automaton with the reference finite state automaton creating alternative paths to form a composed finite state automaton and tracking a number of the alternative paths created. The method further includes pruning the alternative paths based on likely top paths, backtracking over most likely paths of the composed finite state automaton, and rescoring edit-distances of the composed finite state automaton.

Systems and methods for aligning a reference sequence of symbols with hypothesis requiring reduced processing and memory

Conversational understanding systems allow users to conversationally interface with a computing device. In examples, a query may be received that includes a request for execution of a task. A data exchange task definition may be accessed. The data exchange task definition assists a conversational understanding system in managing task state tracking for information needed for task execution. Using the data exchange task definition, a per-turn policy for interacting with the user computing device is generated based on the state of a dialogue with a computing device and an evaluation of a process flow chart provided by a task owner resource. The task owner resource may be independent from the conversational understanding system. A response to the query may be generated and output based on the per-turn policy. In examples, the per-turn policy is used to generate one or more responses during a dialogue with a user via a computing device.

Policy authoring for task state tracking during dialogue

Methods and systems are provided for discriminating ambiguous expressions to enhance user experience. For example, a natural language expression may be received by a speech recognition component. The natural language expression may include at least one of words, terms, and phrases of text. A dialog hypothesis set from the natural language expression may be created by using contextual information. In some cases, the dialog hypothesis set has at least two dialog hypotheses. A plurality of dialog responses may be generated for the dialog hypothesis set. The dialog hypothesis set may be ranked based on an analysis of the plurality of the dialog responses. An action may be performed based on ranking the dialog hypothesis set.

Discriminating ambiguous expressions to enhance user experience

Representative embodiments disclose mechanisms to automatically rank and select extensions triggered in a digital assistant. A sample set of extensions are executed against a set of curated queries in order to extract a set of features and/or statistics. The system trains a machine learning model based on the features and/or statistics to rank and select extensions based on their response to a query. New extension incorporated into the system are executed against a second set of curated queries to obtain a set of extracted features and/or statistics which are saved for use at runtime. At runtime, a query phrase received by the system triggers one or more tasks from extensions. Extracted features for the triggered extensions are combined with stored features/statistics and at least a subset of the results presented to the trained ranking and selection model. The model ranks and selects appropriate tasks which are presented to the user.

Digital assistant extension automatic ranking and selection

Aspects herein provide third party application authors with a user interface authoring platform that automates and simplifies a task definition process while also providing the ability to leverage pre-existing language understanding models and canonicalization and resolution modules that are provided by the operating system on which the CU system resides or as provided by other third parties. In particular, the present disclosure provides a method and system for authoring a task using a user interface authoring platform.

Natural language task completion platform authoring for third party experiences

Non-limiting examples of the present disclosure describe decoupling task state tracking that is managed by a shared task completion platform from execution of tasks by a task resource owner. Task registration data is received at a shared task state platform for a task that is executable by a task owner resource. Task registration data comprises parameters to be collected for execution of the task and ancillary information, such as the name of the task and whether to confirm the values of the parameters after collection. During interaction with a user, the shared task completion platform receives an input and determines the task is associated with the received input. During the interaction, parameters of the received task registration data are utilized to collect data for execution of the task. The collected data is transmitted to the task owner resource for execution of the task.

Task state tracking in systems and services

An arrangement and corresponding method are described for multi-domain natural language processing. Multiple parallel domain pipelines are used for processing a natural language input. Each domain pipeline represents a different specific subject domain of related concepts. Each domain pipeline includes a mention module that processes the natural language input using natural language understanding (NLU) to determine a corresponding list of mentions, and an interpretation generator that receives the list of mentions and produces a rank-ordered domain output set of sentence-level interpretation candidates. A global evidence ranker receives the domain output sets from the domain pipelines and produces an overall rank-ordered final output set of sentence-level interpretations.

Multi-domain natural language processing architecture

In language evaluation systems, user expressions are often evaluated by speech recognizers and language parsers, and among several possible translations, a highest-probability translation is selected and added to a dialog sequence. However, such systems may exhibit inadequacies by discarding alternative translations that may initially exhibit a lower probability, but that may have a higher probability when evaluated in the full context of the dialog, including subsequent expressions. Presented herein are techniques for communicating with a user by formulating a dialog hypothesis set identifying hypothesis probabilities for a set of dialog hypotheses, using generative and/or discriminative models, and repeatedly re-ranks the dialog hypotheses based on subsequent expressions. Additionally, knowledge sources may inform a model-based with a pre-knowledge fetch that facilitates pruning of the hypothesis search space at an early stage, thereby enhancing the accuracy of language parsing while also reducing the latency of the expression evaluation and economizing computing resources.

Dialogue evaluation via multiple hypothesis ranking

An arrangement and corresponding method are described for natural language processing. A natural language understanding (NLU) arrangement processes a natural language input to determine a corresponding sentence-level interpretation. A user state component maintains user context data that characterizes an operating context of the NLU arrangement. Operation of the NLU arrangement is biased by the user context data.

Situation aware NLU/NLP

An automated arrangement is described for conducting natural language interactions with a human user. A user interface is provided for user communication in a given active natural language interaction with a natural language application during an automated dialog session. An automatic speech recognition (ASR) engine processes unknown user speech inputs from the user interface to produce corresponding speech recognition results. A natural language concept module processes the speech recognition results to develop corresponding natural language concept items. A concept item storage holds selected concept items for reuse in a subsequent natural language interaction with the user during the automated dialog session.

Growing community of inventors

Mercier, Canada

Jean-Philippe Robichaud