Nvidia Corporation

Boris Ginsburg

Alex Fit-Florea

Amir Gholaminejad

Nithin Rao Koluguri

Pooya Davoodi

Jagadeesh Balam

Michael Houston

Slawomir Kierat

Vitaly Sergeyevich Lavrukhin

Evelina Bakhturina

Hao Wu

Yang Zhang

Evgenii Shabalin

Roman Korostik

Ahmad Kiswani

Taejin Park

Subhankar Ghosh

Cheng-Ping Hsieh

Sergei Nikolaev

Vladimir Bataev

Disclosed are apparatuses, systems, and techniques that may use machine learning for implementing speaker diarization. The techniques include obtaining a speaker embedding for various reference times of a speech and for various differently-sized time intervals, identifying a plurality of clusters, each cluster associated with a different speaker of the speech. The techniques further include computing, using the speaker embeddings, a set of embedding weights for various differently-sized time intervals, and identifying, using the computed set of the embedding weights, one or more speakers speaking at a respective reference time.

Multi-scale speaker diarization for conversational AI systems and applications

A method, computer readable medium, and system are disclosed for rounding floating point values. Dynamic directional rounding is a rounding technique for floating point operations. A floating point operation (addition, subtraction, multiplication, etc.) is performed on an operand to compute a floating point result. A sign (positive or negative) of the operand is identified. In one embodiment, the sign determines a direction in which the floating point result is rounded (towards negative or positive infinity). When used for updating parameters of a neural network during backpropagation, dynamic directional rounding ensures that rounding is performed in the direction of the gradient.

Dynamic directional rounding

In various examples, first textual data may be applied to a first MLM to generate an intermediate speech representation (e.g., a frequency-domain representation), the intermediate audio representation and a second MLM may be used to generate output data indicating second textual data, and parameters of the second MLM may be updated using the output data and ground truth data associated with the first textual data. The first MLM may include a trained Text-To-Speech (TTS) model and the second MLM may include an Automatic Speech Recognition (ASR) model. A generator from a generative adversarial networks may be used to enhance an initial intermediate audio representation generated using the first MLM and the enhanced intermediate audio representation may be provided to the second MLM. The generator may include generator blocks that receive the initial intermediate audio representation to sequentially generate the enhanced intermediate audio representation.

Hybrid language models for conversational AI systems and applications

Disclosed are apparatuses, systems, and techniques that may use machine learning for implementing speaker recognition, verification, and/or diarization. The techniques include applying a neural network (NN) to a speech data to obtain a speaker embedding representative of an association between the speech data and a speaker that produced the speech. The speech data includes a plurality of frames and a plurality of channels representative of spectral content of the speech data. The NN has one or more blocks of neurons that include a first branch performing convolutions of the speech data across the plurality of channels and across the plurality of frames and a second branch performing convolutions of the speech data across the plurality of channels. Obtained speaker embeddings may be used for various tasks of speaker identification, verification, and/or diarization.

Speaker identification, verification, and diarization using neural networks for conversational AI systems and applications

In various examples, one or more text-to-speech machine learning models may be customized or adapted to accommodate new or additional speakers or speaker voices without requiring a full re-training of the models. For example, a base model may be trained on a set of one or more speakers and, after training or deployment, the model may be adapted to support one or more other speakers. To do this, one or more additional layers (e.g., adapter layers) may be added to the model, and the model may be re-trained or updated—e.g., by freezing parameters of the base model while updating parameters of the adapter layers—to generate an adapted model that can support the one or more original speakers of the base model in addition to the one or more additional speakers corresponding to the adapter layers.

Customizing text-to-speech language models using adapters for conversational AI systems and applications

Aspects of the present invention are directed to computer-implemented techniques for improving the training of artificial neural networks using a reduced precision (e.g., float16) data format. Embodiments of the present invention rescale tensor values prior to performing matrix operations (such as matrix multiplication or matrix addition) to prevent overflow and underflow. To preserve accuracy throughout the performance of the matrix operations, the scale factors are defined using a novel data format to represent tensors, wherein a matrix is represented by the tuple X, where X=(a, v[.]), wherein a is a float scale factor and v[.] are scaled values stored in the float16 format. The value of any element X[i] according to this data format would be equal to a*v[i].

Tensor processing using low precision format

Systems and methods provide for text normalization or inverse text normalization using a hybrid language system that combines rule-based processing with neural or learned processing. For example, a hybrid rule-based and neural approach identifies semiotic tokens within a textual input and generates a set of potential plain-text conversions of the semiotic tokens. The plain-text conversions are weighted and evaluated by a trained language model that rescores the plain-text conversion based on context to identify a highest scoring plain-text conversion for further processing within a language system pipeline.

Text normalization and inverse text normalization using weighted finite-state transducers and neural language models

Growing community of inventors

Mountain View, CA, United States of America

Boris Ginsburg