At&t Intellectual Property I, L.p.

Lukasz Golab

Theodore Donald Johnson

Howard Karloff

Divesh Srivastava

Philip Korn

Oliver Spatscheck

Vladislav Shkapenyuk

Mohammad Taghi Hajiaghayi

Mohammad Hossein Bateni

Avishek Saha

Bei Yu

Barna Saha

Methods, systems, and products characterize consistency of data in a stream warehouse. A warehouse table is derived from a continuously received a stream of data. The warehouse table is stored in memory as a plurality of temporal partitions, with each temporal partition storing data within a contiguous range of time. A level of consistency is assigned to each temporal partition in the warehouse table.

Methods, systems, and products for maintaining data consistency in a stream warehouse

Given a set of data for which a conservation law is an appropriate characterization, 'hold' and/or 'fail' tableaux are provided for the underlying conservation law, thereby providing a conservation dependency whereby portions of the data for which the law approximately holds or fails can be discovered and summarized in a semantically meaningful way.

Conservation dependencies

Methods, systems, and products schedule updates to stream warehouses. A plurality of execution tracks are allocated in memory, with each execution track assigned a portion of utilization of a processor and a proportionate portion of the memory. A home track is also defined in the plurality of execution tracks. When the home track is unavailable, then a different track in the plurality of execution tracks is selected, and a database update job is dispatched to the different track.

Methods, systems, and products for stream warehousing

Data tables in data warehouses are updated to minimize staleness and stretch of the data tables. New data is received from external sources and, in response, update requests are generated. Accumulated update requests may be batched. Data tables may be weighted to affect the order in which update requests are serviced.

Minimizing staleness in real-time data warehouses

The specification describes data processes for analyzing large data steams for target anomalies. 'Sequential dependencies' (SDs) are chosen for ordered data and present a framework for discovering which subsets of the data obey a given sequential dependency. Given an interval G, an SD on attributes X and Y, written as X&#x2192;G Y, denotes that the distance between the Y-values of any two consecutive records, when sorted on X, are within G. SDs may be extended to Conditional Sequential Dependencies (CSDs), consisting of an underlying SD plus a representation of the subsets of the data that satisfy the SD. The conditional approximate sequential dependencies may be expressed as pattern tableaux, i.e., compact representations of the subsets of the data that satisfy the underlying dependency.

Processing data using sequential dependencies

Techniques are disclosed for generating conditional functional dependency (CFD) pattern tableaux having the desirable properties of support, confidence and parsimony. These techniques include both a greedy algorithm for generating a tableau and, for large data sets, an 'on-demand' algorithm that outperforms the basic greedy algorithm in running time by an order of magnitude. In addition, a range tableau, as a generalization of a pattern tableau, can achieve even more parsimony.

Generating conditional functional dependencies

A method for scheduling atomic update jobs to a streaming data warehouse includes allocating execution tracks for executing the update jobs. The tracks may be assigned a portion of available processor utilization and memory. A database table may be associated with a given track. An update job directed to the database table may be dispatched to the given track for the database table, when the track is available. When the track is not available, the update job may be executed on a different track. Furthermore, pending update jobs directed to common database tables may be combined and separated in certain transient conditions.

Method for scheduling updates in a streaming data warehouse

Two methods and computer-readable medium for obtaining information using field group unpacking functions. The first method obtains information using field group unpacking functions by identifying an optimized unpacking function from field group unpacking functions, and an optimized unpacking function is used to unpack a field associated with the data stream. The second method obtains information using field group unpacking functions by identifying an optimized unpacking function from the field group unpacking functions. Then, a prefilter is applied and associated with the optimized unpacking functions and used to unpack a field associated with the data stream. The computer-readable medium obtains field group unpacking functions for execution by a computing device using field group unpacking functions that identify an optimized unpacking function from the field group unpacking functions, and use an optimized unpacking function to unpack a field associated with the data stream.

Optimized field unpacking for a data stream management system

A method and system are disclosed for operating a high speed data stream management system which runs a query plan including a set of queries on a data feed in the form of a stream of tuples. A predicate prefilter is placed outside the query plan upstream of the set of queries, and includes predicates selected from those used by the queries. Predicates are selected for inclusion in the prefilter based on a cost heuristic, and predicates are combined into composites using a rectangle mapping heuristic. The prefilter evaluates the presence of individual and composite predicates in the tuples and returns a bit vector for each tuple with bits representing the presence or absence of predicates in the tuple. A bit signature is assigned to each query to represent the predicates related to that query, and a query is invoked when the tuple bit vector and the query bit signature are compatible.

Growing community of inventors

Morris Plains, NJ, United States of America

Lukasz Golab