International Business Machines Corporation

Ricoh Company, Ltd.

Akihiro Nakayama

Ken Kumagai

Masaki Komedani

Takuma Murakami

Raphael C Polig

Kubilay Atasu

Tong Xu

Kazuhiro Yamada

Shunsuke Ishikawa

Mikhail Sogrin

Kei Sugano

Kazuo Aoki

Susumu Fukuda

Chikako Oyanagi

Alexander Troussov

Hiroshi Inokawa

Takahiro Uchiki

John Judge

Generating rules to automatically extract linguistic patterns from documents is provided. A first plurality of linguistic pattern extraction rules corresponding to a user-selected text example from a document is generated according to a first abstraction rule of a plurality of abstraction rules. Each respective linguistic pattern extraction rule of the first plurality of linguistic pattern extraction rules having a first identified level of abstraction. The first plurality of linguistic pattern extraction rules ordered by the first identified level of abstraction is presented in a first list to a user via a user interface. A selection of one particular linguistic pattern extraction rule is received from the first list by the user via the user interface. That one particular linguistic pattern extraction rule selected by the user is applied to the document to automatically extract user-desired linguistic patterns similar to the user-selected text example from the document.

Controlling abstraction of rule generation based on linguistic context

A system, method, and computer program product for text annotation. The system includes at least one processing component, at least one memory component, an annotation corpus, and a document processor. The document processor is configured to receive a document including at least one annotated text span. The at least one annotated text span is annotated with a type from a type system (e.g., a domain-specific type system). The document processor is also configured to extract linguistic features of the at least one annotated text span, and generate a type attribute for the type based on the extracted linguistic features. Further, the document processor is configured to receive a new annotated text span, which is annotated with the type. The document processor extracts linguistic features of the new annotated text span, and determines whether the linguistic features match the type attribute.

Linguistically consistent document annotation

A method retrains a lexical analysis model. One or more processors divide a token in a document into a first sub-token having a boundary that begins with a start point of a mention and a second sub-token that ends at an end point of the mention. The processor(s) determine that the start/end point of the mention does not coincide with a start/end point of the first sub-token; concatenate the first sub-token with the second sub-token to create a concatenated token; and store the concatenated token in a user dictionary for the document. The processor(s) perform a morphological analysis of the document using the concatenated token from the user dictionary, and retrain a lexical analysis model for the document by leveraging the performing of the morphological analysis of the document.

Adjusting an operation of a system based on a modified lexical analysis model for a document

A cascaded finite-state-transducer array includes a plurality of finite-state-transducers, the finite-state-transducers being distributed in space. The finite-state-transducer array is configured with dedicated data transfer channels between the finite-state-transducers to transfer specific data types. Each data stream on a dedicated data transfer channel may transmit a particular data type, which may be sorted in increasing order of start offsets or token IDs.

Hardware compilation of cascaded grammars

A method for evaluating annotation quality is provided. The method may include obtaining annotation information associated with a plurality of annotators and a plurality of data elements including a plurality of annotation entries corresponding to at least one data element and entered based on an annotation guideline, determining a quality rating for the annotation guideline based on a comparison between a first value associated with the plurality of annotators and the plurality of data elements and a second value associated with any disparity among the plurality of annotation entries, determining a proficiency rating for an annotator from the plurality of annotators based on a comparison between a third value associated with annotation entries by the annotator and the second value, and generating a report based on the quality rating and the proficiency rating.

Evaluating quality of annotation

An alignment of attributes is extracted from a pair of mentions in an annotated corpus. An annotated document is read from the annotated corpus. Three character string arrays (A, B, and C) are stored into a first TRIE. The character string arrays include character string representations of attributes of one or more first tokens, second tokens, and neighborhood tokens of mentions of the annotated document, respectively. The storing into the first TRIE obtains partial integer arrays (K, L, and M) that include identifiers corresponding to the attributes of the mentions. The partial integer arrays are stored into a second TRIE. The storing into the second TRIE obtains an identifier array (X) that includes three identifiers. The identifier array is stored into a third TRIE. The storing into the TRIE obtains one identifier of the alignment of attributes.

Reduction of memory usage in feature generation

A system and method to hardware-accelerate finite state transducer libraries and their compilation toolchains. In an embodiment, a computer-implemented method for partitioning an UIMA-PEAR file into software-based and hardware-accelerated components may comprise creating a data-flow graph representation of the UIMA-PEAR-file, flattening hierarchies of the data-flow graph representation, and selecting the components to be hardware accelerated from the flattened hierarchies of the data-flow graph representation based on data dependencies of data types produced and consumed by each component of the flattened data-flow graph.

Accelerated content analytics based on a hierarchical data-flow-graph representation

An image forming apparatus, which executes printing according to a print request based on an instruction by a user of a client terminal, includes a print time calculation unit that calculates a required time for printing according to the print request; and a display unit that displays a plurality of contents on a display device during an execution of printing in accordance with a content list including the plurality of contents for the user, the content list being generated based on the calculated required time for printing.

Image forming apparatus, image forming system and display control method for displaying content based on a required amount of time for printing

To improve the processing speed of a morphological analysis system, an analysis processing part retrieves words constituting a natural language sentence from a word dictionary, retrieves concatenation rules from a grammar dictionary, creates an automaton based on the words and the concatenation rules, and obtains a morphological analysis. The grammar dictionary stores rules integrating concatenation rules representing non-null transitions and concatenation rules representing null transitions in which transition sources of the first concatenation rule are their transition destinations. To create the automaton, an optimal solution searching part of the analysis processing part generates, responsive to an input word entered into the automaton, states necessary for transition according to the concatenation rules corresponding to the input word without generating any state that can be transited by pursuing a null transition succeeding the generated states.

Apparatus and method for morphological analysis

Method and apparatus for finding the focus of a document. A semantic network includes a plurality of nodes, each representing a concept, and links, connecting the nodes, representing relations between the concepts is used. The method including: providing a list of terms in an input document which match concepts represented by the semantic network, and a frequency value for each matched term indicating the number of times the term appears in the document; mapping each matched term to a referent node or to a plurality of possible referent nodes in the semantic network, and assigning weights to nodes.

Method and system for finding a focus of a document

A regular-expression dictionary, as opposed to an exact-match dictionary, is constructed for utilization within computerized textual analysis. One or more data types are defined for entries within the regular-expression dictionary, within a markup-language schema. A structure in the markup-language schema is provided that encompasses the data types of the entries within the regular-expression dictionary. For each of a number of entries of the regular-expression dictionary, a mapping methodology is defined between a source definition of the entry to the structure and the data types within the markup-language schema. Furthermore, a definition of the entry is mapped to the data types within the structure of the markup-language schema, using the mapping methodology defined. The markup language may be eXtensible Markup Language (XML), or another type of markup language.

Growing community of inventors

Kawasaki, Japan

Akihiro Nakayama