Baidu USA LLC

Shulong Tan

Hongliang Fei

Ping Li

Zhixin Zhou

Zhaozhuo Xu

Wei Fan

Weijie Zhao

Yi Zhen

Chaochun Liu

Yu Long Cao

Nan Du

Richard Chun Ching Wang

Dawen Zhou

Bocong Liu

Retrieval of relevant vectors produced by representation learning can critically influence the efficiency in Natural Language Processing (NLP) tasks. Presented herein are systems and methods for searching vectors via a typical nonmetric matching function: inner product. Embodiments, which construct an approximate Inner Product Delaunay Graph (IPDG) for top-1 Maximum Inner Product Search (MIPS), transform retrieving the most suitable latent vectors into a graph search problem with great benefits of efficiency. Experiments on data representations learned for different machine learning tasks verify the outperforming effectiveness and efficiency of IPDG embodiments.

Efficient retrieval of top similarity representations

Presented are systems and methods that construct BipartitE Graph INdices (BEGIN) embodiments for fast neural ranking. BEGIN embodiments comprise two types of nodes: sampled queries and base or searching objects. In one or more embodiments, edges connecting these nodes are constructed by using a neural network ranking measure. These embodiments extend traditional search-on-graph methods and lend themselves to fast neural ranking. Experimental results demonstrate the effectiveness and efficiency of such embodiments.

Fast neural ranking on bipartite graph indices

Efficient inner product search is important for many data ranking services, such as recommendation and Information Retrieval. Efficient retrieval via inner product dramatically influences the performance of such data searching and retrieval systems. To resolve deficiencies of prior approaches, embodiments of a new index graph construction approach, referred to generally as Norm Adjusted Proximity Graph (NAPG), for approximate Maximum Inner Product Search (MIPS) are presented. With adjusting factors estimated on sampled data, NAPG embodiments select more meaningful data points to connect with when constructing a graph-based index for inner product search. Extensive experiments verify that the improved graph-based index pushes the state-of-the-art of inner product search forward greatly, in the trade-off between search efficiency and effectiveness.

Norm adjusted proximity graph for fast inner product retrieval

Incremental proximity graph maintenance (IPGM) systems and methods for online ANN search support both online vertex deletion and insertion of vertices on proximity graphs. In various embodiments, updating a proximity graph comprises receiving a workload that represents a set of vertices in the proximity graph, each vertex being associated with a type of operation such as a query, insertion, or deletion. For a query or an insertion, a search may be executed on the graph to obtain a set of top-K vertices for each vertex. In the case of a deletion, a vertex may be deleted from the proximity graph, and a local or global reconnection update method may be used to reconstruct at least a portion of the proximity graph.

Proximity graph maintenance for fast online nearest neighbor search

Presented herein are embodiments of a fast search on graph methodology for Maximum Inner Product Search (MIPS). This optimization problem is challenging since traditional Approximate Nearest Neighbor (ANN) search methods may not perform efficiently in the nonmetric similarity measure. Embodiments herein are based on the property that a Möbius/Möbius-like transformation introduces an isomorphism between a subgraph of-Delaunay graph and Delaunay graph for inner product. Under this observation, embodiments of a novel graph indexing and searching methodology are presented to find the optimal solution with the largest inner product with the query. Experiments show significant improvements compared to existing methods.

Transformation for fast inner product search on graph

Due to the high language use variability in real-life, manual construction of semantic resources to cover all synonyms is prohibitively expensive and may result in limited coverage. Described herein are systems and methods that automate the process of synonymy resource development, including both formal entities and noisy descriptions from end-users. Embodiments of a multi-task model with hierarchical task relationship are presented that learn more representative entity/term embeddings and apply them to synonym prediction. In model embodiments, a skip-gram word embedding model is extended by introducing an auxiliary task 'neighboring word/term semantic type prediction' and hierarchically organize them based on the task complexity. In one or more embodiments, existing term-term synonymous knowledge is integrated into the word embedding learning framework. Embeddings trained from the multi-task model embodiments yield significant improvement for entity semantic relatedness evaluation, neighboring word/term semantic type prediction, and synonym prediction compared with baselines.

Hierarchical multi-task term embedding learning for synonym prediction

Presented are systems and methods that allow healthcare providers and governments to infer demand for healthcare resources to ensure effective and timely healthcare services to patients by reducing healthcare supply shortages, emergencies, and healthcare costs. In embodiments, this is accomplished by gathering data from a number of sources to generate labeled records from which entity features and relationships between entities are extracted, correlates, and/or combined with other external healthcare data. In embodiments, this information is used to train a model that predicts healthcare resource demands given a set of input conditions or factors.

Systems and methods for estimating healthcare resource demand

Presented are relation inference methods and systems that use deep learning techniques for data mining documents to discover a relation between terms of interest in a given field covering a specific topic. For example, in the healthcare domain, various embodiments of the present disclosure provide for a relation inference system that mines large-scale medical documents in a free-text database to extract symptom and disease terms and generates relation information that aids in disease diagnosis. In embodiments, this is accomplished by training and using an RNN, such as an LSTM, a Gated Recurrent Unit (GRU), etc., that takes advantage of a term dictionary to examine co-occurrences of terms of interest within documents to discover correlations between the terms. The correlation may then be used to predict statistically most probable terms (e.g., a disease) related to a given search term (e.g., a symptom).

Systems and methods for relation inference

Systems and methods are disclosed to identify entities that have a similar meaning, and may, in embodiments, be grouped into entity groups for knowledge base construction. In embodiments, the entity relations of similarity or non-similarity for an entity pair are predicted as a binary relationship. In embodiments, the prediction may be based upon similarity score between the entities and the entity features, which features are constructed using an entity feature or representation model. In embodiments, the prediction may be an iterative process involving minimum human checking and existing knowledge update. In embodiments, one or more entity groups are formed using graph search from the predicted entity pairs. In embodiments, a group centroid entity may be selected to represent each group based on one or more factors, such as its generality or popularity.

Growing community of inventors

Santa Clara, CA, United States of America

Shulong Tan