At+t Corp.

Amperity, Inc.

University of Washington

Other

Microsoft Technology Licensing, LLC

Dan Suciu

Yan Yan

Stephen Meyles

Carlos Minoru Sakoda

Maria F Fernandez

Divesh Srivastava

Michael P Fikes

Graeme Andrew Kyle Roche

Jeffrey Allen Stokes

Balachander Krishnamurthy

Guy Jacobson

Ian Wesley-Smith

Wang-Chiew Tan

Susan B Davidson

Peter Buneman

Christopher Re

Arvind Arasu

Sihem Amer-Yahia

Alon Yitzchak Levy

Trevor Jim

Nilesh N Dalvi

Atsuyuki Morishima

Jaewoo Kang

Daniela Dina Florescu

The present disclosure relates clustering similar data records together in a hierarchical clustering scheme. Each tier in a cluster corresponds to a minimal match score, which reflects a degree of confidence. In this respect, a higher confidence may lead to smaller sized clusters while a lower confidence may lead to larger sized clusters. Ordinal classification may be used to generate hierarchical clusters. In some embodiments, hierarchical clustering with conflict resolution is used to resolve user-defined hard conflicts in each tier of the clustering results.

Multi-level conflict-free entity clusters

The present disclosure relates to optimizing one or more database tables that may include one or more redundant records. Records are clustered and assigned stable identifiers. In this manner, the underlying records within a cluster are not removed or deleted. As updates to the database are made, new clustering analyses are performed using the underlying records and any updates made. Newly identified clusters are reassigned stable identifiers.

Maintaining stable record identifiers in the presence of updated data records

The present disclosure relates to fuse multiple database tables together. The fields of the database tables may be normalized using semantic fields. Under a first approach, database tables are deduplicated by consolidating redundant records. This may be done by performing pairwise comparisons to identify related pairs of records and then clustering the related pairs of records. Then, the deduplicated database tables are merged by performing another pairwise comparison. Under a second approach, the database tables may be concatenated. Thereafter, records are subject to pairwise comparisons and then clustered to create a merged database table.

Effectively fusing database tables

Multi-level conflict-free entity clusterings

The present disclosure relates clustering similar data records together in a hierarchical clustering scheme. Each tier in a cluster corresponds to a minimal match score, which reflects a degree of confidence. A hierarchical cluster ID is generated for respective data records. The hierarchical cluster ID may be made up of a series of values, wherein each value reflects a tier within the hierarchical clustering scheme. A user may enter a partial hierarchical cluster ID to select clusters associated with a lower confidence. Thus, in some embodiments, the hierarchical cluster ID is variable in length in a manner that corresponds to the tiers in the hierarchical clustering scheme.

Clustering of data records with hierarchical cluster IDs

A system, framework, and algorithms for data deduplication are described. A declarative language, such as a Datalog-type logic language, is provided. Programs in the language describe data to be deduplicated and soft and hard constraints that must/should be satisfied by data deduplicated according to the program. To execute the programs, algorithms for performing graph clustering are described.

Declarative framework for deduplication

A novel approach that computes and efficiently ranks the top-k answers to a query on a probabilistic database. The approach identifies the top-k answers, since imprecisions in the data often lead to a large number of answers of low quality. The algorithm is used to run several Monte Carlo simulations in parallel, one for each candidate answer, and approximates the probability of each only to the extent needed to correctly determine the top-k answers. The algorithm is provably optimal and scales to large databases. A more general application can identify a number of top-rated entities of a group that satisfy a condition, based on a criteria or score computed for the entities. Also disclosed are several optimization techniques. One option is to rank the top-rated results; another option provides for interrupting the iteration to return the number of top-rated entities that have thus far been identified.

Efficient top-K query evaluation on probabilistic data

A system that supports arbitrarily complex SQL queries with 'uncertain' predicates. The query semantics are based on a probabilistic model and the results are ranked, much like in Information Retrieval, based upon their probability. An optimization algorithm is employed that can efficiently compute most queries. The algorithm attempts to determine whether a proposed plan is a safe plan that can be used for correctly evaluating the query. Operators such as the project operator in the proposed plan are evaluated to determine if they are safe. If so, the proposed plan is safe and will produce correct answers in a result. Due to the data complexity of some queries, a safe plan may not exist for a query. For these queries, either a 'least unsafe plan,' or a Monte-Carlo simulation algorithm can be employed to produce a result with answers that have an acceptable error.

Computing probabilistic answers to queries

The occurrence of recursive loops between network elements are detected and prevented. One or more queries are generated that are sent between the network elements. One or more of the network elements detect the imminent occurrence of a recursive loop between the network elements, and prevent the recursive loop by generating an intensional answer in response to the query. The intensional answer contains rules.

Method for prevention of recursive loops between network elements

A technique for performing query evaluation on distributed directories utilizes the creation of a 'topology cache' defining the hierarchical relationship between the various directory servers (i.e., identifying 'subordinate' and &#x201c;superior&#x201d; knowledge references associated with each directory server in the system). The created topology cache is then stored at each directory server, and forwarded to a client upon submitting a query to the system. Using the topology cache information at the client, a distributed query evaluation plan can be developed for use with complex queries, such as hierarchical and aggregate queries.

Distributed evalulation of directory queries using a topology cache

A method for converting relational data to XML (eXtensible Markup Language) is provided. The method can use a greedy algorithm to efficiently construct materialized XML views of relational databases. A greedy algorithm designed for XML view definition queries is provided for decomposing a large query into smaller queries and determining which query will run faster without actually running the query.

Method for converting relational data into XML

A method for converting relational data to XML (Extensible Markup Language) is provided. The method, sometimes referred to as SilkRoute, provides a general, dynamic and efficient tool for viewing and querying relational data in XML. SilkRoute can express mappings of relational data in XML that conforms to arbitrary public document type definitions. Also, SilkRoute can materialize the fragment of an XML view needed by an application and it can fully exploit the query engine of a relational database management system whenever data items in an XML view need to be materialized.

Growing community of inventors

Seattle, WA, United States of America

Dan Suciu