International Business Machines Corporation

The Johns Hopkins University

Baidu USA LLC

Nuance Communications, Inc.

Kenneth W Church

Bhuvana Ramabhadran

Abhinav Sethy

Vaibhava Goel

Samuel Jonathan Thomas

Etienne Marcheret

Lidia Mangu

Stanley S Chen

Laurence P Sansone

Masayuki A Suzuki

Gakuto Kurata

Ryuki Tachibana

Jason William Pelecanos

Dimitrios B Dimitriadis

Petr Fousek

Weizhong Zhu

George Andrei Saon

Hynek Hermansky

Miroslav Novak

Keith Kintzley

Aren Jansen

Ashish Verma

Martin Franz

Jeffrey N Marcus

Shajith I Mohamed

Jeffrey Scott McCarley

Prasanta Kumar Ghosh

Nicholas Stephen Kersting

Josef Vopicka

Hee Woo Jun

Joel Hestness

Zheng Zu

Described herein are systems and methods for word embeddings to avoid the need to throw out rare words appearing less than a certain number of times in a corpus. Embodiments of the present disclosure involve group words into clusters/classes for multiple times using different assignments of the vocabulary words to a number of classes. Multiple copies of the training corpus are then generated using the assignments to replace each word with the appropriate class. A word embedding generating model is run on the multiple class corpora to generate multiple class embeddings. An estimate of the gold word embedding matrix is then reconstructed from multiple pairs of assignments, class embeddings, and covariances. Test results show the effectiveness of embodiments of the present disclosure.

Embeddings with classes

Symbol sequences are estimated using a computer-implemented method including detecting one or more candidates of a target symbol sequence from a speech-to-text data, extracting a related portion of each candidate from the speech-to-text data, detecting repetition of at least a partial sequence of each candidate within the related portion of the corresponding candidate, labeling the detected repetition with a repetition indication, and estimating whether each candidate is the target symbol sequence, using the corresponding related portion including the repetition indication of each of the candidates.

Symbol sequence estimation in speech

An approach is provided that receives an audio stream and utilizes a voice activation detection (VAD) process to create a digital audio stream of voices from at least two different speakers. An automatic speech recognition (ASR) process is applied to the digital stream with the ASR process resulting in the spoken words to which a speaker turn detection (STD) process is applied to identify a number of speaker segments with each speaker segment ending at a word boundary. A speaker clustering algorithm is then applied to the speaker segments to associate one of the speakers with each of the speaker segments.

Diarization driven by the ASR based segmentation

A method of combining data streams from fixed audio-visual sensors with data streams from personal mobile devices including, forming a communication link with at least one of one or more personal mobile devices; receiving at least one of an audio data stream and/or a video data stream from the at least one of the one or more personal mobile devices; determining the quality of the at least one of the audio data stream and/or the video data stream, wherein the audio data stream and/or the video data stream having a quality above a threshold quality is retained; and combining the retained audio data stream and/or the video data stream with the data streams from the fixed audio-visual sensors.

Combining installed audio-visual sensors with ad-hoc mobile audio-visual sensors for smart meeting rooms

It is desirable to have audio and/or video systems and processing tools that can automatically record audio/video and analyze such recordings to capture material that may be relevant to a user. In one or more embodiments disclosed herein, a recording may be condensed by using one or more tools, including but not limited to, converting speech to text and searching for relevant content, keywords, and the like; detecting and sorting speakers and/or content (e.g., events in a conversation); removing non-substantive content (e.g., silences and other irrelevant content); adjusting the audio for increased playback speed; using prosodic and other indicia in the audio to identify areas of interest; performing diarization; using pseudo-random or random sample to select content; and other methods to extract information to provide a summary or representation of recorded content for review by a user.

Systems and methods for consolidating recorded content

A diarization embodiment may include a system that clusters data up to a current point in time and consolidates it with the past decisions, and then returns the result that minimizes the difference with past decisions. The consolidation may be achieved by performing a permutation of the different possible labels and comparing the distance. For speaker diarization, a distance may be determined based on a minimum edit or hamming distance. The distance may alternatively be a measure other than the minimum edit or hamming distance. The clustering may have a finite time window over which the analysis is performed.

Prefix methods for diarization in streaming mode

An approach is provided that receives an audio stream and utilizes a voice activation detection (VAD) process to create a digital audio stream of voices from at least two different speakers. An automatic speech recognition (ASR) process is applied to the digital stream with the ASR process resulting in the spoken words to which a speaker turn detection (STD) process is applied to identify a number of speaker segments with each speaker segment ending at a word boundary. The STD process analyzes a number of speaker segments using a language model that determines when speaker changes occur. A speaker clustering algorithm is then applied to the speaker segments to associate one of the speakers with each of the speaker segments.

Diarization driven by meta-information identified in discussion content

Identification of an intent of a conversation can be useful for real-time or post-processing purposes. According to example embodiments, a method, and corresponding apparatus of identifying at least one intent-bearing utterance in a conversation, comprises determining at least one feature for each utterance among a subset of utterances of the conversation; classifying each utterance among the subset of utterances, using a classifier, as an intent classification or a non-intent classification based at least in part on a subset of the at least one determined feature; and selecting at least one utterance, with intent classification, as an intent-bearing utterance based at least in part on classification results by the classifier. Through identification of an intent bearing utterance, a call center for example, can provide improved service for callers through, for example, more effective directing of a call to a live agent.

System and method for identification of intent segment(s) in caller-agent conversations

Embodiments of the present invention provide a computer-implemented method for communicating a reference code for a transaction. The method monitors a communication session conducted between a user and an agent via a communication channel, extracts user and channel properties from the monitored communication session, selects a reference code from a set of references codes stored on a database, in which the selection is based at least in part on the extracted communication channel properties and the extracted user properties, and then communicates the selected reference code to the user.

Adaptive selection of message data properties for improving communication throughput and reliability

Embodiments of the invention include method, systems and computer program products for role modeling. Aspects of the invention include receiving, by a processor, audio data, wherein the audio data includes a plurality of audio conversation for one or more speakers. The one or more segments for each of the plurality of audio conversations are partitioned. A speaker is associated with each of the one or more segments. The one or more segments for each of the plurality of audio conversations are labeled with roles utilizing a speaker recognition engine. Speakers are clustered based at least in part on a number of times the speakers are present in an audio conversation.

Role modeling in call centers and work centers

A system and method are provided for performing speech processing. A system includes an audio detection system configured to receive a signal including speech and a memory having stored therein a database of keyword models forming an ensemble of filters associated with each keyword in the database. A processor is configured to receive the signal including speech from the audio detection system, decompose the signal including speech into a sparse set of phonetic impulses, and access the database of keywords and convolve the sparse set of phonetic impulses with the ensemble of filters. The processor is further configured to identify keywords within the signal including speech based a result of the convolution and control operation the electronic system based on the keywords identified.

System and method for processing speech to identify keywords or other information

Growing community of inventors

Croton-on-Hudson, NY, United States of America

Kenneth W Church