Xerox Corporation

Naver Corporation

Conduent Business Services, LLC

Centre National De La Recherche Scientifique

Marc Dymetman

Nicola Cancedda

Jean-Luc Meunier

Max Copperman

Sriram Venkatapathy

Caroline Brun

Christer E Fernstrom

Guillaume M Bouchard

Jean-Michel Renders

Eric Gaussier

Cyril Goutte

Hady Elsahar

Claire Gardent

Gabriela Csurka

Antonietta Grasso

Chunyang Xiao

Wilker Ferreira Aziz

Stéphane Clinchant

Kenji Yamada

Jean-Pierre Chanod

Natalie Glance

Gregory Alan Hanneman

Ioan Calapodescu

Vassilina Nikoulina

Dave Snowdon

Frederique Segond

Shachar Mirkin

Michel Simard

Jean-Marc Coursimault

Pierre Isabelle

Tetiana Parshakova

German Kruszewski

Christophe Servan

Jean-Marc Andréoli

Bruno Cavestro

Mikhail Zaslavskiy

Nadi Tomeh

Ergun Bicici

Spandana Gella

Aurelien Max

Frederic Tendeau

Lucia Specia

Vu Cong Duy Hoang

Muhammad Khalifa

Bryan Eikema

Raghav Goyal

Prasanth Kolachina

Benjamin Roth

Phong Le

Andrew R McCallum

A sampling system includes: an energy-based model (EBM) configured to generate non-negative scores of an input having discrete classifications, respectively; and a sampling module configured to: generate a sample from a probability distribution of the EBM using a proposal distribution; set a probability of acceptance of the sample based on a minimum of (a) 1 and (b) an acceptance value determined based on the sample, a score of the sample from the EBM, the proposal distribution, and an upper boundary value; determine a distribution value between 0 and 1 using a uniform distribution; and discard the sample when the distribution value is greater than the probability of acceptance of the sample.

Sampling from discrete energy-based models with quality/efficiency trade-off

A method for generating a language model for text generation by receiving a pre-trained language model having attributes with existing probability distributions over the pre-trained language model; receiving at least one target constraint; the target constraint specifying an expectation of a target attribute over a language model that approximates the pre-trained language model; computing parameters of an energy based model by applying the target constraint to the pre-trained language model; obtaining samples from a reference policy; updating parameters of a target policy using the obtained samples and the energy based model; updating the reference policy with the target policy if the target policy is superior to the reference policy; and outputting the target policy as a target language model. The target language model is adapted to generate text with the target attribute over a probability distribution that approximates the desired probability distribution specified by the target constraint.

Method and system for controlling distributions of attributes in language models for text generation

In a method for generating a normalized sequential model using a processor, a sequential energy-based model computed by a parameterized neural network is provided. The sequential energy-based model defines an unnormalized probability distribution over a target sequence for a context source. The normalized sequential model is generated by projecting the sequential energy-based model onto a target autoregressive model that approximates a normalized distribution associated with the sequential energy-based model.

Methods and systems for producing neural sequential models

Methods for training a neural sequence-to-sequence (seq2seq) model. A processor receives the model and training data comprising a plurality of training source sequences and corresponding training target sequences, and generates corresponding predicted target sequences. Model parameters are updated based on a comparison of predicted target sequences to training target sequences to reduce or minimize both a local loss in the predicted target sequences and an expected loss of one or more global or semantic features or constraints between the predicted target sequences and the training target sequences given the training source sequences. Expected loss is based on global or semantic features or constraints of general target sequences given general source sequences.

Method and system for training neural sequence-to-sequence models by incorporating global features

Methods, systems, and devices for semantic parsing. In an example embodiment, a method for semantic parsing can include steps, operations, or instructions such as obtaining a data pair for learning, the data pair comprising logical form data and natural utterance data; acquiring grammar for targeted logical forms among the logical form data of the data pair; modeling data comprising other available prior knowledge utilizing WFSA (Weighted Finite State Automata); combining with the targeted logical forms with the data modeled comprising the other available prior knowledge to form a background; and exploiting the background on the data pair. Note that we do not 'learn' the background, but 'learn' the background-RNN (Recurrent Neural Network).

Symbolic priors for recurrent neural network based semantic parsing

A dialog device comprises a natural language interfacing device (chat interface or a telephonic device), and a natural language output device (the chat interface, a display device, or a speech synthesizer outputting to the telephonic device). A computer stores natural language dialog conducted via the interfacing device and constructs a current utterance word-by-word. Each word is chosen by applying a plurality of language models to a context comprising concatenation of the stored dialog and the current utterance thus far. Each language model outputs a distribution over the words of a vocabulary. A recurrent neural network (RNN) is applied to the distributions to generate a mixture distribution. The next word is chosen using the mixture distribution. The output device outputs the current natural language utterance after it has been constructed by the computer.

Dialog device with dialog support generated using a mixture of language models combined using a recurrent neural network

A method and a system for generating a target character sequence from a semantic representation including a sequence of characters are provided. The method includes adapting a target background model, built from a vocabulary of words, to form an adapted background model. The adapted background model accepts subsequences of an input semantic representation as well as words from the vocabulary. The input semantic representation is represented as a sequence of character embeddings, which are input to an encoder. The encoder encodes each of the character embeddings to generate a respective character representation. A decoder then generates a target sequence of characters, based on the set of character representations. At a plurality of time steps, a next character in the target sequence is selected as a function of a previously generated character(s) of the target sequence and the adapted background model.

Natural language generation through character-based recurrent neural networks with finite-state prior knowledge

A method for predicting a canonical form for an input text sequence includes predicting the canonical form with a neural network model. The model includes an encoder, which generates a first representation of the input text sequence based on a representation of n-grams in the text sequence and a second representation of the input text sequence generated by a first neural network. The model also includes a decoder which sequentially predicts terms of the canonical form based on the first and second representations and a predicted prefix of the canonical form. The canonical form can be used, for example, to query a knowledge base or to generate a next utterance in a discourse.

Semantic parsing using deep neural networks for predicting canonical forms

A system and method are provided which employ a neural network model which has been trained to predict a sequentialized form for an input text sequence. The sequentialized form includes a sequence of symbols. The neural network model includes an encoder which generates a representation of the input text sequence based on a representation of n-grams in the text sequence and a decoder which sequentially predicts a next symbol of the sequentialized form based on the representation and a predicted prefix of the sequentialized form. Given an input text sequence, a sequentialized form is predicted with the trained neural network model. The sequentialized form is converted to a structured form and information based on the structured form is output.

Sequence-based structured prediction for semantic parsing

In rejection sampling of a function or distribution p over a space X, a proposal distribution qis refined responsive to rejection of a sample x*&#x3b5;X to generate a refined proposal distribution qselected to satisfy the criteria p(x)&#x2266;q(x)&#x2266;q(x) and q(x*)&#x3c;q(x*). In a sampling mode, the sample x* is obtained by random sampling of the space X, the rejection sampling accepts or rejects x* based on comparison of a ratio p(x*)/q(x*) with a random draw, and the refined proposal distribution qis selected to minimize a norm &#x2225;q&#x2225;where &#x3b1;&#x3c;&#x221e;. In an optimization mode, the sample x* is obtained such that q*=q(x*) maximizes qover the space X, the rejection sampling accepts or rejects x* based on a difference between or ratio of q* and p(x*), and the refined proposal distribution qis selected to minimize a norm &#x2225;q&#x2225;=max{q(x)}.

Joint algorithm for sampling and optimization and natural language processing applications of same

A system and method are disclosed which enable more effective email response authoring by contact center agents, for example, by automatically suggesting prototypical (entire) email responses to the human agent and interactive suggestion of next sentence candidates during the writing process. In one method, a customer inquiry is received and a latent topic prediction is generated, based on a word-based representation of the customer inquiry. A latent topic prediction is generated for an entire agent's reply to the customer inquiry as a function of the latent topic prediction generated for the customer inquiry. A further latent topic prediction is generated for a next sentence of the agent's reply as a function of a topic prediction for the next sentence which is generated with a prediction model that has been trained on annotated sentences of agent replies. Information is output to assist the agent, based on the topic predictions.

Method and system for assisting contact center agents in composing electronic mail replies

A system and a method for phrase-based translation are disclosed. The method includes receiving source language text to be translated into target language text. One or more dynamic bi-phrases are generated, based on the source text and the application of one or more rules, which may be based on user descriptions. A dynamic feature value is associated with each of the dynamic bi-phrases. For a sentence of the source text, static bi-phrases are retrieved from a bi-phrase table, each of the static bi-phrases being associated with one or more values of static features. Any of the dynamic bi-phrases which each cover at least one word of the source text are also retrieved, which together form a set of active bi-phrases. Translation hypotheses are generated using active bi-phrases from the set and scored with a translation scoring model which takes into account the static and dynamic feature values of the bi-phrases used in the respective hypothesis. A translation, based on the hypothesis scores, is then output.

Dynamic bi-phrases for statistical machine translation

Agent utterances are generated for implementing dialog acts recommended by a dialog manager of a call center. To this end, a set of word lattices, each represented as a weighted finite state automaton (WFSA), is constructed from training dialogs between call center agents and second parties (e.g. customers). The word lattices are assigned conditional probabilities over dialog act type. For each dialog act received from the dialog manager, the word lattices are ranked by the conditional probabilities for the dialog act type. At least one word lattice is chosen from the ranking, and is instantiated to generate a recommended agent utterance for implementing the recommended dialog act. The word lattices may be constructed by clustering agent utterances of training dialogs using context features from preceding second party utterances and grammatical dependency link features between words within agent utterances. Path variations of the word lattices may define slots or paraphrases.

Learning generation templates from dialog transcripts

Each entry of an ARPA table for a modeled language includes an n-gram Az, an associated backoff value Az.p equal to the conditional probability p(z|A) that symbol z follows context A in the modeled language, and an associated backoff weight value Az.b for the context A. A method comprises: (1) computing and adding for each entry of the ARPA table in descending n-gram order an associated maximum backoff weight product value Az.m; (2) after performing operation (1), computing and adding for each entry of the ARPA table in descending n-gram order an associated max-backoff value Az.w=maxp(z|hA) which is the maximum backoff value for any head h preceding the context A of the n-gram Az; and (3) extending the ARPA table by adding a column storing the associated maximum backoff weight product values Az.m and a column storing the associated max-backoff values Az.w.

Procedure for building a max-ARPA table in order to compute optimistic back-offs in a language model

A method for terminological adaptation includes receiving a vocabulary pair including source and target language terms. Each term is in a class which includes a set of sequences. Contextual phrase pairs are extracted from a bilingual training corpus, each including source and target phrases. The phrases each include a sequence of the same class as the respective source and target terms as well as some associated context. Templates are generated, based on the contextual phrase pairs. In each template the source and target sequences of a contextual phrase pair are replaced with respective placeholders, each denoting the respective class of the sequence. Candidate phrase pairs are generated from these templates. In each candidate phrase pair, the placeholders of one of the templates are replaced with respective terms of a vocabulary pair of the same class. Some candidate phrase pairs are incorporated into a phrase table of a machine translation system.

Terminological adaptation of statistical machine translation system through automatic generation of phrasal contexts for bilingual terms

Growing community of inventors

Grenoble, France

Marc Dymetman