Samsung Electronics Co., Ltd.

Vijendra R Apsingekar

Taeyeon Ki

Myungjong Kim

Srinivasa Rao Ponakala

Cindy Sushen Tseng

Anil Sunder Yadav

Mohammad Mahdi Moazzami

Divya Neelagiri

Pu Song

Sivakumar Balasubramanian

Asif Ali

Sungjae Park

Hyuk Jun Oh

Aviral Anshu

Gowtham Srinivasan

Akash Sahoo

SeungBeom Ryu

A method includes obtaining audio data and identifying an utterance of a wake word or phrase in the audio data. The method also includes generating an embedding vector based on the utterance from the audio data and accessing a set of previously-generated vectors representing previous utterances of the wake word or phrase. The method further includes performing clustering on the embedding vector and the set of previously-generated vectors to identify a cluster including the embedding vector, where the identified cluster is associated with a speaker. The method also includes updating a speaker vector associated with the speaker based on the embedding vector and determining, using a speaker verification model, a similarity score between the updated speaker vector and the embedding vector. In addition, the method includes determining, based on the similarity score, whether a speaker providing the utterance matches the speaker associated with the identified cluster.

System and method for speaker verification for voice assistant

A method includes obtaining an audio input and a location associated with an electronic device. The method also includes generating an audio embedding associated with the audio input. The method further includes determining a first difference between the audio embedding associated with the audio input and an audio embedding associated with a known user. The method also includes determining a second difference between the location associated with the electronic device and a known location associated with the known user. The method further includes generating, using a false trigger mitigation (FTM) system, a probability of the audio input including a false trigger for automatic speech recognition based on the audio input, the first difference, and the second difference. In addition, the method includes determining whether to perform automatic speech recognition based on the probability.

Context-aware false trigger mitigation for automatic speech recognition (ASR) systems or other systems

A method includes obtaining video content and associated substantially mono audio content. The method also includes determining at least one of a position or a motion trajectory of each of one or more objects detected in the video content and classifying each of the one or more objects into one of multiple object classes. The method further includes separating audio streams within the audio content based on the video content. Each of the audio streams is associated with one of multiple audio sources. The method also includes classifying each of the audio sources into one of the object classes. In addition, the method includes, for each audio source classified into the same object class as one of the one or more objects, distributing the audio stream associated with that audio source into multiple audio channels based on at least one of the position or the motion trajectory of that object.

Three-dimensional (3D) sound rendering with multi-channel audio based on mono audio input

A method includes obtaining at least a portion of an audio stream containing speech activity. At least the portion of the audio stream includes multiple segments. The method also includes, for each of the segments, generating an embedding vector that represents the segment. The method further includes, within each of multiple local windows, clustering the embedding vectors into one or more clusters to perform speaker identification. Different clusters correspond to different speakers. The method also includes presenting at least one first sequence of speaker identities based on the speaker identification for the local windows. The method further includes, within each of multiple global windows, clustering the embedding vectors into one or more clusters to perform speaker identification. Each global window includes two or more of the local windows. The method further includes presenting at least one second sequence of speaker identities based on the speaker identification for the global windows.

Online speaker diarization using local and global clustering

A method includes obtaining an audio input of a person speaking, where the audio input is captured by an electronic device. The method also includes, for each of multiple language types, (i) determining a first probability that the person is speaking in the language type by applying a trained spoken language identification model to the audio input, (ii) determining at least one second probability that the person is speaking in the language type based on at least one characteristic of the person or the electronic device, and (iii) determining a score for the language type based on a weighted sum of the first and second probabilities. The method further includes identifying the language type associated with a highest score as a spoken language of the person in the audio input.

Personalized multi-modal spoken language identification

A method includes accessing, using at least one processor of an electronic device, a machine learning model. The machine learning model is a trained student model that is trained using audio samples in a plurality of accent types. The method also includes receiving, using the at least one processor, an audio input from an audio input device. The method further includes providing, using the at least one processor, the audio input to the trained student model. The method also includes receiving, using the at least one processor, an output from the trained student model including frame-level probabilities associated with the audio input. In addition, the method includes instructing, using the at least one processor, at least one action based on the frame-level probabilities associated with the audio input.

System and method for accent-agnostic frame-level wake word detection

A method includes training a set of teacher models. Training the set of teacher models includes, for each individual teacher model of the set of teacher models, training the individual teacher model to transcribe unlabeled audio samples and predict a pseudo labeled dataset having multiple labels. At least some of the unlabeled audio samples contain named entity (NE) audio data. At least some of the labels include transcribed NE labels corresponding to the NE audio data. The method also includes correcting at least some of the transcribed NE labels using user-specific NE textual data. The method further includes retraining the set of teacher models based on the pseudo labeled dataset from a selected one of the teacher models, where the selected one of the teacher models predicts the pseudo labeled dataset more accurately than other teacher models of the set of teacher models.

System and method for improving named entity recognition

An apparatus for processing speech data may include a processor configured to: separate an input speech into speech signals; identify a bandwidth of each of the speech signals; extract speaker embeddings from the speech signals based on the bandwidth of each of the speech signals, using at least one neural network configured to receive the speech signals and output the speaker embeddings; and cluster the speaker embeddings into one or more speaker clusters, each speaker cluster corresponding to a speaker identity.

Method and apparatus for performing speaker diarization on mixed-bandwidth speech signals

A method includes performing, using at least one processor, feature extraction of input audio data to identify extracted features associated with the input audio data. The method also includes detecting, using the at least one processor, a language associated with the input audio data by processing the extracted features using a plurality of language models, where each language model is associated with a different language. The method further includes directing, using the at least one processor, the input audio data to one of a plurality of automatic speech recognition (ASR) models based on the language associated with the input audio data.

System and method for multi-spoken language detection

A method includes performing, using at least one processor, feature extraction of input audio data to identify extracted features associated with the input audio data. The method also includes detecting, using the at least one processor, a language associated with each of multiple portions of the input audio data by processing the extracted features using a plurality of language models, where each language model is associated with a different language. In addition, the method includes directing, using the at least one processor, each portion of the input audio data to one of a plurality of automatic speech recognition (ASR) models based on the language associated with the portion of the input audio data.

Growing community of inventors

San Jose, CA, United States of America

Vijendra R Apsingekar