International Business Machines Corporation

Doron Cohen

Haggai Roitman

David Konopnicki

Yosi Mass

Shai Erera

Guy Feigenblat

Odellia Boni

David Carmel

Michal Jacovi

Bar Weiner

Oren Sar-Shalom

Yoelle Maarek-Smadja

Nir Levine

Eitan D Farchi

Oded Margalit

Yoelle Smadja Maarek

Avraham Harpaz

Jatin Ganhotra

Michael Herscovici

Aya Soffer

Alan Hartman

Dafna Sheinwald

Ronald Fagin

Michal Shmueli-Scheuer

Mila Keren

Vladimir Soroka

Benjamin Sznajder

Sigalit Ur

Or Rivlin

Gadi Aleksandrowicz

Mel Shalev

An example system includes a processor to receive a pseudo-relevance set including top results form a search engine in response to transmitting a set of concatenated messages of a dialog. The processor can execute a first fixed point operation on the pseudo-relevance set to generate weighted terms. The processor can also execute a second fixed point operation on a message graph including nodes with a heaviness based on the weighted terms.

Ranking of messages in dialogs using fixed point operations

Clarification-question selection, including: Receiving a search conversation that includes utterances by a user and by a conversational search system. Retrieving, from a solution documents database, text passages that are relevant to the search conversation. Retrieving, from a clarification questions database, for each of the text passages, candidate clarification questions that are relevant to both the respective text passage and the search conversation. Ranking the candidate clarification questions according to (a) a first deep learning model that outputs, for each of the candidate clarification questions, a score denoting a strength of association between the respective candidate clarification question and the search conversation, and/or (b) a second deep learning model that outputs, for each of the candidate clarification questions, a score denoting a strength of association between the respective candidate clarification question, the search conversation, and the text passage that was retrieved for the respective candidate clarification question.

Clarification questions selection for conversational search, trained from human-to-human conversations and solution documents

A method, computer system, and a computer program product for natural language processing are provided. A first text corpus that includes semi-structured content that includes hierarchical nodes may be received. Some of the hierarchical nodes may be masked. Node embeddings and level embeddings may be generated from the semi-structured content of the first text corpus and from the masked hierarchical nodes. The node embeddings and the level embeddings may be included in a bi-directional transformer model. The bi-directional transformer model may be trained on the first text corpus by reducing loss from the bi-directional transformer model predicting the masked hierarchical nodes.

Semi-structured content aware bi-directional transformer

Training a machine learning language model to generate clarification questions for use in conversational search, including: Obtaining multiple dialogs between users and agents, each dialog including messages exchanged between a user and an agent, wherein one of the messages of each dialog includes a reference to a solution document provided by the agent. For each of the dialogs, operating a search engine to retrieve a text passage, relevant to at least one of the messages of the respective dialog, from the respective solution document. Training a machine learning language model to generate a new clarification question given at least one new message and multiple new text passages, wherein the training is based on a training set which comprises, for each of the dialogs: said at least one of the messages of the respective dialog, and the text passage retrieved for the respective dialog.

Automatic generation of clarification questions for conversational search

A computer-implemented method, computerized apparatus and computer program product for minimum coordination passage scoring. Given a candidate passage in a document collection potentially matching a query received, a set of overlapping terms between the candidate passage and the query is determined. For each overlapping term in the set, a first measure of a weight of the term in the query, a second measure of a weight of the term in the candidate passage, and a third measure of a specificity of the term in the document collection are calculated. a function of the first and second measure is evaluated to obtain a value reflecting a condition on the relation therebetween. A minimum coordination score representing a relative similarity between the candidate passage and the query is determined based on the value and the first, second and third measures obtained for each of the overlapping terms.

Minimum coordination passage scoring

A computer-implemented method, computerized apparatus and computer program product for query performance prediction, the method comprising: obtaining a result list comprising a listing of documents retrieved from a collection in response to a query; obtaining for each of the listed documents in the result list a score indicating a measure of the document's relevance to the query; sampling the result list to obtain a plurality of sub-lists each of which comprising a listing of documents subsumed by the result list; for each of the plurality of sub-lists, analyzing scores of the documents listed therein to obtain a sample performance estimator; and estimating performance of the result list based on the sample performance estimator of each of the plurality of sub-lists.

Method and system for query performance prediction

Automated keyphrase extraction from a digital text document. A pool of candidate keyphrases of the digital text document is created. A cross-entropy method is then employed to compute a set of output keyphrases out of the pool of candidate keyphrases, by iteratively optimizing an objective function that is configured to cause the set of output keyphrases to be descriptive of one or more main topics discussed in the digital text document. The set of output keyphrases may be used for at least one of: text summarization, text categorization, opinion mining, and document indexing.

Automatic keyphrase extraction from text using the cross-entropy method

A method including: Obtaining multiple conversation texts, one text per conversation, wherein each of the multiple conversation texts comprises: multiple messages authored by multiple parties, and a reference to an electronic document that provides resolution of a problem that is common to all the conversations. Calculating an importance score for each of the multiple messages of all the conversation texts. Clustering the multiple messages of all the conversation texts into multiple bins. Calculating an aggregated importance score for each of the multiple bins, based on the importance scores of the messages contained in the respective bin. Enriching (a) the electronic document, or (b) a record of the electronic document in an index of electronic documents, with at least some of the multiple bins and their aggregated importance scores, wherein the at least some of the multiple bins are added as fields to the electronic document or to the record.

Document enrichment with conversation texts, for enhanced information retrieval

A method for automated selection of a search result ranker comprising: providing a set of queries; for each of said queries, receiving, from a search engine, a plurality of relevancy score sets, wherein each relevancy score set is associated with search results found in a corpus of electronic documents using each of a plurality of computerized search result rankers; calculating a difficulty score for each of said queries relative to all other queries in the set, based on said plurality of relevancy score sets associated with said query; calculating a quality score for each of said search result rankers based on said plurality of relevancy score sets associated with said search result ranker, wherein each of said plurality of relevancy score sets is weighed according to the difficulty score of its associated query; and selecting one of said search rankers based on said quality score.

Automated selection of search ranker

A method comprising receiving digital documents, a query statement, and a summary length constraint; identifying, for each of said digital documents, a sentence subset, based, at least in part, on said query statement, a modified version of said summary length constraint, and a first set of quality objectives, generating, for each of said sentence subsets, a random forest representation; iteratively (i) sampling, from each of said random forest representations, a plurality of tokens to create a corresponding candidate document summary, based, at least in part, on weights assigned to each of said tokens, (ii) assigning a quality ranking to said candidate document summary, based, at least in part, on said first set of quality objectives and a second set of quality objectives, and (iii) adjusting said weights, based, at least in part, on said quality rankings; and outputting a highest ranking said candidate document as a compressed summary.

Context-aware sentence compression

Embodiments may provide automated summarization of documents, such as scientific documents by using a prior distribution on logical sections learnt from a corpus of human authored summaries. For example, a method of document summarization may comprise receiving, at the computer system, a document and segmenting the document into a plurality of sentences, identifying, at the computer system, sections in the document and aligning each sentence in the document to a section logical role, and summarizing, at the computer system, the document using a probability distribution.

Unsupervised attention based scientific document summarization

A session search relevance model identifies a user's dynamic information need based on a feedback model and a session relevance model. The feedback model is based on query changes in the session search and user interest in particular documents presented throughout the session search. The relevance model modifies a user's current query to retrieve documents most relevant to a user's information need.

Relevance model for session search

A computerized method comprising using hardware processor(s) for receiving, from a computerized search engine, digital input data comprising a group of relevancy score sets, where each relevancy score set comprises scores associated with computerized search terms and search field pairs found in electronic documents. Two or more statistical values are computed of the relevancy score sets, one or more of the two or more statistical values for each relevancy score set. Based on some of the two or more statistical values, some relevancy scores sets are reduced from the group to create a reduced group. The reduced group is sent to the computerized search engine for presenting a search result to a user on a computer display.

Multi-field search query ranking using scoring statistics

A computer-implemented method, computerized apparatus and computer program product for minimum coordination passage scoring. Given a candidate passage in a document collection potentially matching a query received, a set of overlapping terms between the candidate passage and the query is determined. For each overlapping term in the set, a first measure of a weight of the term in the query, a second measure of a weight of the term in the candidate passage, and a third measure of a specificity of the term in the document collection are calculated. A function of the first and second measure is evaluated to obtain a value reflecting a condition on the relation therebetween. A minimum coordination score representing a relative similarity between the candidate passage and the query is determined based on the value and the first, second and third measures obtained for each of the overlapping terms.

Growing community of inventors

Gilon, Israel

Doron Cohen