Facebook, Inc.

Carnegie Mellon University

Meta Platforms, Inc.

Other

Honda Motor Co., Ltd.

Mobile Technologies, LLC

Accumente, LLC

Ian R Lane

Alexander Waibel

Jike Chong

Jungsuk Kim

Rakesh Gupta

Antoine Roland Raux

Senaka Buthpitiya

A speech translation system and methods for cross-lingual communication that enable users to improve and customize content and usage of the system and easily. The methods include, in response to receiving an utterance including a first term associated with a field, translating the utterance into a second language. In response to receiving an indication to add the first term associated with the field to a first recognition lexicon, adding the first term associated with the field and the determined translation to a first machine translation module and to a shared database for a community associated with the field of the first term associated with the field, wherein the first term associated with the field added to the shared database is accessible by the community.

Lexicon development via shared translation database

Disclosed herein is method of performing speech recognition using audio and visual information, where the visual information provides data related to a person's face. Image preprocessing identifies regions of interest, which is then combined with the audio data before being processed by a speech recognition engine.

System and method for audio-visual speech recognition

Disclosed herein is a GPU-accelerated speech recognition engine optimized for faster than real time speech recognition on a scalable server-client heterogeneous CPU-GPU architecture, which is specifically optimized to simultaneously decode multiple users in real-time. In order to efficiently support real-time speech recognition for multiple users, a 'producer/consumer' design pattern is applied to decouple speech processes that run at different rates in order to handle multiple processes at the same time. Furthermore, the speech recognition process is divided into multiple consumers in order to maximize hardware utilization. As a result, the platform architecture is able to process more than 45 real-time audio streams with an average latency of less than 0.3 seconds using one-million-word vocabulary language models.

System and method for multi-user GPU-accelerated speech recognition engine for client-server architectures

The present invention describes methods for performing large-scale graph traversal calculations on parallel processor platforms. The invention describes methods for on-the-fly hypothesis rescoring that utilizes graphic processing units (GPUs) in combination with utilizing central processing units (CPUs) of computing devices. The invention is described in one embodiment as applied to the task of large vocabulary continuous speech recognition.

Methods for hybrid GPU/CPU data processing

A method and system for multimodal human-vehicle interaction including receiving input from an occupant in a vehicle via more than one mode and performing multimodal recognition of the input. The method also includes augmenting at least one recognition hypothesis based on at least one visual point of interest and determining a belief state of the occupant's intent based on the recognition hypothesis. The method further includes selecting an action to take based on the determined belief state.

System and method for multimodal human-vehicle interaction and belief tracking

A field maintainable class-based translation system and apparatus with components that ease use by linguistically untrained users is disclosed. The apparatus includes modules for recovering errors, extending and customizing language coverage and increasing the speed of effective communication.

Speech translation with back-channeling cues

A speech translation system and methods for cross-lingual communication that enable users to improve and modify content and usage of the system and easily abort or reset translation. The system includes a speech recognition module configured for accepting an utterance and adding a new word, a machine translation module, an interface configured to communicate the utterance and proposed translation, a correction module and an abort action unit that removes any hypotheses or partial hypotheses and terminates translation. The system also includes modules for storing favorites, changing language mode, automatically identifying language, providing language drills, viewing third party information relevant to conversation, among other things.

Enhanced speech-to-speech translation system and methods for adding a new word

A method of optimizing the calculation of matching scores between phone states and acoustic frames across a matrix of an expected progression of phone states aligned with an observed progression of acoustic frames within an utterance is provided. The matrix has a plurality of cells associated with a characteristic acoustic frame and a characteristic phone state. A first set and second set of cells that meet a threshold probability of matching a first phone state or a second phone state, respectively, are determined. The phone states are stored on a local cache of a first core and a second core, respectively. The first and second sets of cells are also provided to the first core and second core, respectively. Further, matching scores of each characteristic state and characteristic observation of each cell of the first set of cells and of the second set of cells are calculated.

Utilizing multiple processing units for rapid training of hidden markov models

A method and apparatus are provided for updating the vocabulary of a speech translation system for translating a first language into a second language including written and spoken words. The method includes adding a new word in the first language to a first recognition lexicon of the first language and associating a description with the new word, wherein the description contains pronunciation and word class information. The new word and description are then updated in a first machine translation module associated with the first language. The first machine translation module contains a first tagging module, a first translation model and a first language module, and is configured to translate the new word to a corresponding translated word in the second language. Optionally, the invention may be used for bidirectional or multi-directional translation.

Growing community of inventors

Sunnyvale, CA, United States of America

Ian R Lane