International Business Machines Corporation

Robert T Ward

Salim Estephan Roukos

Jiri Navratil

Mark Edward Epstein

Radu Florian

Martin Franz

Xiaoqiang Luo

Abraham P Ittycheriah

Laura Chiticariu

Hiroshi Kanayama

Jeffrey Scott McCarley

Kishore A Papineni

Stephen A Della Pietra

Paul S Cohen

Leiming R Qian

Srinivasa Patibandla Rao

Jian-Ming Xu

Takuya Ohko

Mohamed Kamal Omar

Sriram Ganapathy

Kishore Ananda Papineni

Joshua David Koppelman

Efsun Kayi

Yousef El-Kurdi

Sharon B Jones

An approach for generating synthetic treebanks to be used in training a parser in a production system is provided. A processor receives a request to generate one or more synthetic treebanks from a production system, wherein the request indicates a language for the one or more synthetic treebanks. A processor retrieves at least one corpus of text in which the requested language is present. A processor provides the at least one corpus to a transformer enhanced parser neural network model. A processor generates at least one synthetic treebank associated with a string of text from the at least one corpus of text in which the requested language is present. A processor sends the at least one synthetic treebank to the production system, wherein the production system trains a parser utilized by the production system with the at least one synthetic treebank.

Treebank synthesis for training production parsers

Exemplary embodiments of the present invention provide a method of system combination in an audio analytics application including providing a plurality of language identification systems in which each of the language identification systems includes a plurality of probabilities. Each probability is associated with the system's ability to detect a particular language. The method of system combination in the audio analytics application includes receiving data at the language identification systems. The received data is different from data used to train the language identification systems. A confidence measure is determined for each of the language identification systems. The confidence measure identifies which language its system predicts for the received data and combining the language identification systems according to the confidence measures.

Method for system combination in an audio analytics application

According to an aspect, a first word in a first language and a second word in a second language in a bilingual corpus are stemmed. A probability for aligning the first stem and the second stem and a distance metric between the normalized first stem and the normalized second stem are calculated. The first word and the second word are identified as a cognate pair when the probability and the distance metric meet a threshold criterion and stored as a cognate pair in a set of cognates. A candidate sentence in the second language is retrieved from a corpus. The candidate sentence is filtered by the active vocabulary of a user in the second language and the set of cognates. A sentence quality score is calculated for the candidate sentence; and the candidate sentence is ranked for presentation to the user based on the sentence quality scorer.

Automatic cognate detection in a computer-assisted language learning system

Techniques for translating and presenting text in a structured document are provided. A structured document including text to be translated is received. At least one portion of the text to be translated is translated into at least one representation of translated text of a specific language. The translation is carried out in accordance with a prioritization schedule. Further, the representation of translated text is provided by at least first and second servers, the first server independently providing the representation of translated text of a first quality and the second server independently providing the representation of translated text of a second quality, the first quality being different than the second quality. As the text is translated, the representation of translated text is presented immediately when available.

Prioritized incremental asynchronous machine translation of structured documents

An improved method, apparatus, and computer instructions displaying search results. A query is received in a first language. A snippet in the document containing the word is selected in response to identifying a document in a second language in which the document contains a word matching a query word in the query. This snippet is translated into the first language.

Display of results of cross language search

A parser is provided that parses a Chinese text stream at the character level and builds a syntactic structure of Chinese character sequences. A character-based syntactic parse tree contains word boundaries, part-of-speech tags, and phrasal structure information. Syntactic knowledge constrains the system when it determines word boundaries. A deterministic procedure is used to convert word-based parse trees into character-based trees. Character-level tags are derived from word-level part-of-speech tags and word-boundary information is encoded with a positional tag. Word-level parts-of-speech become a constituent label in character-based trees. A maximum entropy parser is then built and tested.

Chinese character-based parser

An arrangement for adapting statistical parsers to new data using a mathematical transform, particularly a Markov transform. In particular, it is assumed that an initial statistical parser is available and a batch of new data is given. The initial model is mapped to a new model by a Markov matrix, each of whose rows sums to one. In the unsupervised setup, where 'true' parses are missing, the transform matrix is obtained by maximizing the log likelihood of the parses of test data decoded using the model before adaptation. The proposed algorithm can be applied to supervised adaptation, as well.

Adaptation of statistical parsers based on mathematical transform

A method of developing natural language understanding (NLU) applications can include determining NLU interpretation information from an NLU training corpus of text using a multi-pass processing technique. The alteration of one pass automatically can alter an input for a subsequent pass. The NLU interpretation information can specify an interpretation of at least part of the NLU training corpus of text. The NLU interpretation information can be stored in a database, and selected items of the NLU interpretation information can be presented in a graphical editor. User specified edits also can be received in the graphical editor.

Integrated development tool for building a natural language understanding application

A method for designating a subvocabulary for speech recognition systems includes the steps of providing a vocabulary of words each having a flag with a first value, selecting words to be eliminated from the vocabulary, setting the flags of the selected words to a second value and processing speech based on words having the flag set to the first value. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for disabling and enabling of subvocabularies in speech recognition systems, the method steps include providing a vocabulary of words each having a flag with a first value, selecting words to be eliminated from the vocabulary, setting the flags of the selected words to a second value and processing speech based on words having the flag set to the first value.

Disabling and enabling of subvocabularies in speech recognition systems

A system for conversant interaction includes a recognizer for receiving and processing input information and outputting a recognized representation of the input information. A dialog manager is coupled to the recognizer for receiving the recognized representation of the input information, the dialog manager having task-oriented forms for associating user input information therewith, the dialog manager being capable of selecting an applicable form from the task-oriented forms responsive to the input information by scoring the forms relative to each other. A synthesizer is employed for converting a response generated by the dialog manager to output the response. A program storage device and method are also provided.

Natural language task-oriented dialog manager and method

A system for translating a first word set in a source language into a second word set in a target language, the system comprising: input means for inputting the first word set into the system; tagging means for tagging the first word set input to the system so as to at least substantially reduce non-essential variability in the first word set; translation means including a single a posteriori conditional probability model and a target candidate store for storing target language candidate word sets, wherein the translation means employs the single model to evaluate the target language candidate word sets in order to select the target language candidate word set having a best score with respect to the first word set; and output means for outputting the best scoring target language candidate word set as the second word set in the target language.

Statistical translation system with features based on phrases or groups

The invention proposes using statistical methods to do natural language understanding. The key notion is that there are 'strings' of words in the natural language, that correspond to a single semantic concept. One can then define an alignment between an entire semantic meaning (consisting of a set of semantic concepts), and the English. This is modeled using P(E,A.vertline.S). One can model p(S) separately. This allows each parameter to be modeled using many different statistical models.

Growing community of inventors

Croton-on-Hudson, NY, United States of America

Robert T Ward