Xerox Corporation

Jean-Marc Coursimault

Nicolas Monet

Nidhi Singh

Marc Dymetman

Nicola Cancedda

Herve Poirier

Vassilina Nikoulina

Wilker Ferreira Aziz

Lucia Specia

Herve Poirer

Methods and systems for dynamically training a web crawler. The web crawler maintains one or more categories each comprising a set of words. The method includes selecting at least one hyperlink in response to a query received from a user. The method further includes determining a hyperlink score for the at least one hyperlink based on a category score associated with each of one or more categories. The category score associated with each of the one or more categories is updated based at least in part on the hyperlink score. The updated category score is compared with the hyperlink score to select a category from the one or more categories. The set of words associated with the category is updated based on content of a web page pointed by the at least one hyperlink.

Systems and methods of web crawling

A system and a method for phrase-based translation are disclosed. The method includes receiving source language text to be translated into target language text. One or more dynamic bi-phrases are generated, based on the source text and the application of one or more rules, which may be based on user descriptions. A dynamic feature value is associated with each of the dynamic bi-phrases. For a sentence of the source text, static bi-phrases are retrieved from a bi-phrase table, each of the static bi-phrases being associated with one or more values of static features. Any of the dynamic bi-phrases which each cover at least one word of the source text are also retrieved, which together form a set of active bi-phrases. Translation hypotheses are generated using active bi-phrases from the set and scored with a translation scoring model which takes into account the static and dynamic feature values of the bi-phrases used in the respective hypothesis. A translation, based on the hypothesis scores, is then output.

Dynamic bi-phrases for statistical machine translation

A web page identified by a URL stored in a downloads queue is downloaded, and hyperlinks in the downloaded web page are identified. Each hyperlink is screened by parsing the hyperlink (optionally only the URL of the hyperlink) to identify features comprising character strings, computing for each feature values for one or more meta-features indicative of the hyperlinked web page being in a target language, aggregating the meta-feature values to generate a score for the hyperlink, and adding the URL of the hyperlink to the downloads queue conditional upon the score satisfying a screening criterion. The downloading, identifying, and screening are iteratively repeated to perform web crawling, and an index of web pages in the target language is constructed based on analysis of content of the downloaded web pages. The meta-features may include a transliterated target word meta-feature, a language code meta-feature, a country code meta-feature, or so forth.

Language-oriented focused crawling using transliteration based meta-features

Growing community of inventors

Revel, France

Jean-Marc Coursimault