International Business Machines Corporation

Haggai Roitman

Doron Cohen

Guy Feigenblat

David Konopnicki

Yosi Mass

Shai Erera

Bar Weiner

David Carmel

Odellia Boni

Roy Levin

Roee Shraga

Gilad Barkai

Mustafa Canim

Elad Yom-Tov

Ella Rabinovich

Oren Sar-Shalom

Sivan Yogev

Shay Hummel

Michal Shmueli-Scheuer

Or Rivlin

Nir Levine

Oded Margalit

Mattias Marder

Ido Guy

Artem Barger

Ariel Raviv

Gal Ashour

Boaz Carmeli

Jatin Ganhotra

Dafna Sheinwald

Yossi Mesika

Jonathan Herzig

Yuri G Rabinovich

Anat Hashavit

Nadav Har'El

Erel Uziel

Gadi Aleksandrowicz

Nadav Har'El Yosef

Yonatan Maman

Edward Moroshko

An example system includes a processor to receive a pseudo-relevance set including top results form a search engine in response to transmitting a set of concatenated messages of a dialog. The processor can execute a first fixed point operation on the pseudo-relevance set to generate weighted terms. The processor can also execute a second fixed point operation on a message graph including nodes with a heaviness based on the weighted terms.

Ranking of messages in dialogs using fixed point operations

A system and a computer-implemented method for ranking tabular data entities by likelihood of comprising answers for (natural language) queries, based on multimodal descriptions of the tabular data entities, comprising separate representations, which represent different aspects of the tabular data entities. The ranking is based on joint representations, generated from the query representation and separate representations of the tabular data entities' aspects, using gated multimodal units. The computer-implemented method may be used for applications such as web searches, data aggregation, and research tasks.

Method and system for table retrieval using multimodal deep co-learning with helper query-dependent and query-independent relevance labels

An example system includes a processor to receive concepts extracted from a result set corresponding to a query and result associations for each extracted concept. The processor is to build a graph based on the extracted concepts, wherein the graph comprises a number of nodes representing the extracted concepts and weighted edges representing similarity between concepts extracted from shared results. The processor is to partition the graph into subgraphs with vertices corresponding to candidate facets for vertices having higher sums of weighted edges. The processor is to rank the candidate facets. The processor is to select higher ranked candidate facets to use as facets. The processor is to output facets with a result set in response to the query.

Dynamically generating facets using graph partitioning

An illustrative embodiment includes a method for post-retrieval query performance prediction using hybrid document-passage information. The method includes: obtaining a set of documents responsive to a specific query; extracting document-level information regarding respective documents within the set; extracting passage-level information regarding respective passages of documents within the set; and estimating a likelihood that the set of documents includes relevant information to the specific query using both the document-level information and the passage-level information.

Extended query performance prediction framework utilizing passage-level information

A method, computer system, and a computer program product for natural language processing are provided. A first text corpus that includes semi-structured content that includes hierarchical nodes may be received. Some of the hierarchical nodes may be masked. Node embeddings and level embeddings may be generated from the semi-structured content of the first text corpus and from the masked hierarchical nodes. The node embeddings and the level embeddings may be included in a bi-directional transformer model. The bi-directional transformer model may be trained on the first text corpus by reducing loss from the bi-directional transformer model predicting the masked hierarchical nodes.

Semi-structured content aware bi-directional transformer

An example system includes a processor to receive a query. The processor can retrieve ranked candidates from an index based on the query. The processor can re-rank the ranked candidates using a Bidirectional Encoder Representations from Transformers (BERT) query-question (Q-q) model trained to match queries to questions of a frequently asked question (FAQ) dataset, wherein the BERT Q-q model is fine-tuned using paraphrases generated for the questions in the FAQ dataset. The processor can return the re-ranked candidates in response to the query.

Frequently asked questions and document retrieval using bidirectional encoder representations from transformers (BERT) model trained on generated paraphrases

Training a machine learning language model to generate clarification questions for use in conversational search, including: Obtaining multiple dialogs between users and agents, each dialog including messages exchanged between a user and an agent, wherein one of the messages of each dialog includes a reference to a solution document provided by the agent. For each of the dialogs, operating a search engine to retrieve a text passage, relevant to at least one of the messages of the respective dialog, from the respective solution document. Training a machine learning language model to generate a new clarification question given at least one new message and multiple new text passages, wherein the training is based on a training set which comprises, for each of the dialogs: said at least one of the messages of the respective dialog, and the text passage retrieved for the respective dialog.

Automatic generation of clarification questions for conversational search

Multimodal table encoding, including: Receiving an electronic document that contains a table. The table includes multiple rows, multiple columns, and a schema comprising column labels or row labels. The electronic document includes a description of the table which is located externally to the table. Next, operating separate machine learning encoders to separately encode the description, schema, each of the rows, and each of the columns of the table, respectively. The schema, the rows, and the columns are encoded together with end-of-column tokens and end-of-row tokens that mark an end of each column and row, respectively. Then, applying a machine learning gating mechanism to the encoded description, encoded schema, encoded rows, and encoded columns, to produce a fused encoding of the table, wherein the fused encoding is representative of both a structure of the table and a content of the table.

Multimodal table encoding for information retrieval systems

Ad-hoc table retrieval, including: Representing each of a plurality of tables as a multi-field text document in which: different modalities of the table are represented as separate fields, and a concatenation of all the modalities is represented as a separate, auxiliary field. Receiving a query. Executing the query on the multi-field text documents, to retrieve a list of preliminarily-ranked candidate tables out of the plurality of tables. Calculating an intrinsic table similarity score for each of the candidate tables, based on the query and the auxiliary field. Calculating an extrinsic table similarity score for each of the candidate tables, based on a cluster hypothesis of the candidate tables. Combining: the preliminary rankings, the intrinsic table similarity scores, and the extrinsic table similarity scores, to re-rank the candidate tables.

Table indexing and retrieval using intrinsic and extrinsic table similarity measures

An embodiment for predicting a performance of a query in retrieving multifield documents is provided. The embodiment may include receiving a query from a user. The embodiment may also include retrieving a list of multifield documents from a corpus of documents in response to the query. The embodiment may further include generating a pseudo-effective (PE) reference-list for each field in the corpus of documents. The embodiment may also include executing one or more existing query performance prediction (QPP) methods on the retrieved list and each generated PE reference-list. The embodiment may further include deriving one or more extended QPP methods. The embodiment may also include estimating a performance of the query in obtaining the retrieved list of multifield documents based on the one or more extended QPP methods.

Query performance prediction for multifield document retrieval

A computer-implemented method, computerized apparatus and computer program product for minimum coordination passage scoring. Given a candidate passage in a document collection potentially matching a query received, a set of overlapping terms between the candidate passage and the query is determined. For each overlapping term in the set, a first measure of a weight of the term in the query, a second measure of a weight of the term in the candidate passage, and a third measure of a specificity of the term in the document collection are calculated. a function of the first and second measure is evaluated to obtain a value reflecting a condition on the relation therebetween. A minimum coordination score representing a relative similarity between the candidate passage and the query is determined based on the value and the first, second and third measures obtained for each of the overlapping terms.

Minimum coordination passage scoring

A computer-implemented method, computerized apparatus and computer program product for query performance prediction, the method comprising: obtaining a result list comprising a listing of documents retrieved from a collection in response to a query; obtaining for each of the listed documents in the result list a score indicating a measure of the document's relevance to the query; sampling the result list to obtain a plurality of sub-lists each of which comprising a listing of documents subsumed by the result list; for each of the plurality of sub-lists, analyzing scores of the documents listed therein to obtain a sample performance estimator; and estimating performance of the result list based on the sample performance estimator of each of the plurality of sub-lists.

Method and system for query performance prediction

In a computerized information retrieval system: executing a search based on a query, to retrieve a set of tables ranked according to their relevancy to the query, wherein each of the tables includes one or more columns; selecting, from the retrieved tables, a predefined number of highest-ranking tables; scoring each column in the highest-ranking tables using a link analysis algorithm, and selecting, from the scored columns, a predefined number of highest-scoring columns; scoring terms contained within each of the highest-scoring columns, and selecting, from the scored terms, a predefined number of highest-scoring terms; re-ranking the highest-ranking tables by using the highest-scoring terms as pseudo relevance feedback that expands the query; and providing, as a response to the query, at least one of: the re-ranked tables, ordered according to the re-ranking, and data contained in at least one of the re-ranked tables, wherein the data are ordered according to the re-ranking.

Column-based query expansion for table retrieval

An exemplary method includes: determining a pool of documents, wherein each document is within at least one of a plurality of lists, each of the lists results from executing a query on a corpus, and the corpus comprises at least the pool of documents; determining a first ranking of documents within the pool of documents based at least in part on first scores computed for respective documents within the pool; estimating relevance to the specified query at least of respective documents within the first ranking, wherein the relevance is estimated without user feedback regarding the relevance; and determining a second ranking of documents within the pool based at least in part on second scores computed at least for respective documents within the first ranking, wherein the second score for a given document is computed based at least in part on the estimated relevance of at least the given document.

Growing community of inventors

Yoknea'm Elit, Israel

Haggai Roitman