International Business Machines Corporation

George Andrei Mihaila

Lipyeow Lim

Bishwaranjan Bhattacharjee

Ioana Roxana Stanoi

Sriram Kolinjivadi Padmanabhan

Pawan Raghunath Chowdhary

Nianjun Zhou

Dikran S Meliksetian

Min Wang

Themis Palpanas

Christian Alexander Lang

Hui Lei

Mustafa Canim

Timothy Ray Malkemus

Philip Shilung Yu

Yuan-Chi Chang

Jimeng Sun

Feifei Li

Spyridon Papadimitriou

Oded Shmueli

Rajesh Ramkrishna Bordawekar

Yi Chen

Kenneth Andrew Ross

Sherman Lau

Cathy McArthur

Reza Sherkat

Zoltan G Toth

A method, information processing system, and computer program storage product optimize the placement of database objects on a multiplicity of storage devices. A set of database objects are placed on a first storage device in a multiplicity of storage devices. Each storage device comprises differing characteristics. A query workload is run on the set of database objects that have been placed on the first storage device. Profiling information associated with the query workload that is running is collected. A subset of database objects is selected from the set of the database objects to be stored on a second storage device. The subset of database objects is stored on the second storage device and all remaining database objects in the set of database objects on the first storage device.

Managing database object placement on multiple storage devices

A method, information processing system, and computer program storage product optimize the placement of database objects on a multiplicity of storage devices. A set of database objects are placed on a first storage device in a multiplicity of storage devices. Each storage device comprises differing characteristics. A query workload is run on the set of database objects that have been placed on the first storage device. Profiling information associated with the query workload that is running is collected. A subset of database objects is selected from the set of the database objects to be stored on a second storage device. The second storage device is a separate physical device from, and performs faster than, the first storage device. The subset of database objects is stored on the second storage device and all remaining database objects in the set of database objects on the first storage device.

A method, system, and computer readable medium for collecting statistics associated with data in a database are disclosed. The computer readable medium implements the method comprises determining an amount of memory needed to collect statistics for data associated with a defined data type in a relational database. The defined data type is based upon a mark-up language using a tree structure with one or more root-to-node paths therein. The amount of memory is allocated as determined for collecting the statistics for the data of the defined data type. A statistics collection is performed for the data of the defined data type in a single pass through the database and within the amount of memory which has been allocated. The performing includes at least determining a total number of instances of at least one path-identifier associated with a given value within a given set of documents.

Statistics collection using path-value pairs for relational databases

Techniques are disclosed for selecting a delete-safe compression method for a plurality of delta encoded data values (e.g., delta encoded integers or deltas). For example, a computer-implemented method for selecting an optimal delete-safe compression algorithm from among two or more compression algorithms for use on a plurality of delta encoded data values includes the following steps. The maximum number of data values eliminated by each of the two or more compression algorithms is computed. For the plurality of delta encoded data values to be compressed, the minimum size of the plurality of delta encoded data values before compression thereof is computed. A delete-safe threshold value is computed based on the minimum size of the plurality of delta encoded data values. Then, the compression algorithm is selected from the two or more compression algorithms that achieves the delete-safe threshold value.

Method and apparatus for selecting an optimal delete-safe compression method on list of delta encoded integers

A system, method, and computer readable medium for preserving information in time variant data schemas are disclosed. The method includes determining if at least one modification request associated with a database schema has been received. In response to the modification request being received, a metadata table associated with the database schema is updated to include at least one entry associated with the modification request. The entry identifies an instance in time when an action associated with the modification request was performed.

Management of time-variant data schemas in data warehouses

A method, information processing system, and computer readable medium for modifying at least one data warehouse schema based on detected changes in an associated observation model are disclosed. The method includes determining if at least one new observation model has been created. The method also includes determining if at least one existing observation model is associated with the new observation model. In response to the existing observation model being associated with the new observation model, at least one changed attribute is identified by comparing the new observation model and the existing observation model. A set of files associated with the existing observation model is updated to reflect the changed attribute between the new observation model and the existing observation model.

Updating a data warehouse schema based on changes in an observation model

Techniques for estimating items (e.g., data item or objects) frequencies in large data sets are disclosed. For example, a technique for determining items and their frequencies at multiple levels of interest in a collection of nested bags includes the following steps. A hierarchy of a plurality of levels of nested bags and the levels of interest are inputted. Among the plurality of levels, a subset of bags is sampled from at least one level. At each level of interest, the frequency is counted of each distinct item in the bags obtained in the sampling step. At each level of interest, the item frequencies obtained in the counting step are extrapolated based on sampling ratios associated with the sampling step. At each level of interest, the items are sorted according to their frequencies obtained from the extrapolating step and those items with highest frequencies are retained. A bag may refer to one or more subsets or groups of data items or objects. Also, a bag may, itself, contain one or more other bags.

Techniques for estimating item frequencies in large data sets

Illustrative embodiments provide a computer-implemented method for hybrid push/pull of continuous structured query language queries. The computer-implemented method receives stream input, wherein the stream input comprises events of interest, builds and a state machine and stream plans, based on an original query, and replicates the stream input. Responsive to a push sub-query trigger, the computer-implemented method submits a pull sub-query to the database to produce a result, and sends the result to a requester.

Hybrid push/pull execution of continuous SQL queries

Providing consistency guarantees in a data stream processing engine is provided. Consistency tracking information is attached to data streams coming into the data stream processing engine. The consistency tracking information is propagated through a plurality of streaming operators that process the data streams within the data stream processing engine. Then, the propagated consistency tracking information is used to detect a consistent state in an output stream.

Providing consistency in processing data streams

A method, information processing system, and computer program storage product to compress sorted values. At least a first prefix and a second prefix in a plurality of prefixes are compared. Each prefix comprises at least a portion of a plurality of sorted values. A respective prefix comprises a set of consecutive characters including at least a first character of a respective sorted value. The respective sorted value further comprising a respective suffix comprising consecutive characters of the respective sorted value that are after the respective prefix. At least a respective first character of the first prefix and a respective first character of the second prefix are determined to be substantially identical. The first prefix is merged with the second prefix into a single prefix comprising the first character. A set of suffixes associated with the first prefix is updated to reflect an association with the second prefix.

Compression of sorted value indexes using common prefixes

Disclosed are a system, method, and computer readable medium for collecting statistics associated with data in a database. The method comprises determining an amount of memory needed to collect statistics for data associated with a defined data type in a relational database. The defined data type is based upon a mark-up language using a tree structure with one or more root-to-node paths therein. The amount of memory as determined is allocated for collecting the statistics for the data of the defined data type. A statistics collection is performed for the data of the defined data type in a single pass through the database and within the amount of memory which has been allocated.

Statistics collection using path-identifiers for relational databases

A system and method for managing a cache includes monitoring a temperature of regions on a secondary storage based on a cumulative cost to access pages from each region of the secondary storage. Similar temperature pages are grouped in logical blocks. Data is written to a cache in a logical block granularity by overwriting cooler blocks with hotter blocks.

Temperature-aware buffered caching for solid state storage

Techniques are disclosed for encoding a variable length structure such that it facilitates forward and reverse scans of a list of such structures as needed. While the techniques are applicable to a wide variety of applications, they are particularly well-suited for use with structures such as those found in compressed database indexes. For example, a computer-implemented method for processing one or more variable length data structures includes the following steps. Each variable length data structure is obtained. Each variable length structure comprises one or more data block. A variable length encoding process is applied to the one or more blocks of each variable length data structure which comprises setting a continuation data value in each block to a first value or a second value, wherein the setting of the continuation data values enables bi-directional scanning of each variable length structure.

Method and apparatus for encoding list of variable length structures to support bi-directional scans

A system, method, and computer readable medium for optimizing throughput of a stream processing system are disclosed. The method comprises analyzing a set of input streams and creating, based on the analyzing, an input profile for at least one input stream in the set of input streams. The input profile comprises at least a set of processing requirements associated with the input stream. The method also comprises generating a search space, based on an initial configuration, comprising a plurality of configurations associated with the input stream. A configuration in the plurality of configurations is identified that increases throughput more than the other configurations in the plurality of configurations based on at least one of the input profile and system resources.

Growing community of inventors

Yorktown Heights, NY, United States of America

George Andrei Mihaila