Facebook, Inc.

Mobile Technologies, LLC

Meta Platforms Technologies, LLC

Christian Fuegen

Alexander Waibel

Ying Zhang

Jing Zheng

Guillaume Lample

Naomi Aoki Waibel

Thilo Wolfgang Koehler

Baiyang Liu

Anuj Kumar

Kay Rottmann

Kay Rottman

Yang Gao

Yongquiang Wang

Dmitrii Serdiuk

Weiyi Zheng

Zhaojun Yang

Qing He

In one embodiment, a method includes receiving a voice input from a user and determining a first style of the voice input, based on first features extracted from the voice input. A second style for a voice response having second features may then be determined based on the first style. Finally, the voice response may be generated based on the second features of the second style, and this voice response may be provided in response to the voice input.

Stylizing text-to-speech (TTS) voice response for assistant systems

Exemplary embodiments relate to improvements in spoken language understanding (SLU) systems. Conventionally, SLU systems include an automatic speech recognition (ASR) component configured to receive an input of audio data and to generate a textual representation of the audio data. Conventional SLU systems also include a natural language understanding (NLU) component configured to receive a text-based transcript and perform language-based tasks such as domain classification, intent determination, and slot-filling. However, these two components are typically trained separately based on different metrics. In real-world situations, errors in the ASR component propagate to the NLU component, which degrades the performance of the overall system. Exemplary embodiments described herein perform SLU in an end-to-end manner that infers semantic meaning directly from audio features without an intermediate text representation. This may allow for more a more accurate translation performed in a more resource-efficient manner (particularly in terms of processing resources).

Methods and systems for performing end-to-end spoken language analysis

Components of language processing engines, such as translation models and language models, can be customized for groups of users or based on user type values. Users can be organized into groups or assigned a value on a continuum based on factors such as interests, biographical characteristics, social media interactions, etc. In some implementations, translation engine components can be customized for groups of users by selecting the training data from content created by users in that group. In some implementations, the group identifier or continuum value can be part of the input into a general translation component allowing the translation component to take a language style of that user group into account when performing language processing tasks.

Social hash for language models

A hybrid speech translation system whereby a wireless-enabled client computing device can, in an offline mode, translate input speech utterances from one language to another locally, and also, in an online mode when there is wireless network connectivity, have a remote computer perform the translation and transmit it back to the client computing device via the wireless network for audible outputting by client computing device. The user of the client computing device can transition between modes or the transition can be automatic based on user preferences or settings. The back-end speech translation server system can adapt the various recognition and translation models used by the client computing device in the offline mode based on analysis of user data over time, to thereby configure the client computing device with scaled-down, yet more efficient and faster, models than the back-end speech translation server system, while still be adapted for the user's domain.

Hybrid, offline/online speech translation system

A social networking system obtains user pronunciations of terms whose pronunciations might vary among different users, such as names of users. The social networking system additionally obtains demographic information about the users from whom the pronunciations were obtained, as well as social graph information for those users, such as information about connections of those users in the social graph. Based on the obtained pronunciations, the demographic information, and the social graph information, the social networking system determines, for a user having that name (or other term in question), one or more suggested pronunciations for the name that are likely to be the pronunciations that that user would use.

User-specific pronunciations in a social networking system

A real-time open domain speech translation system for simultaneous translation of a spoken presentation that is a spoken monologue comprising one of a lecture, a speech, a presentation, a colloquium, and a seminar. The system includes an automatic speech recognition unit configured for accepting sound comprising the spoken presentation in a first language and for continuously creating word hypotheses, and a machine translation unit that receives the hypotheses, wherein the machine translation unit outputs a translation, into a second language, from the spoken presentation.

Simultaneous translation of open domain lectures and speeches

Growing community of inventors

Sunnyvale, CA, United States of America

Christian Fuegen