International Business Machines Corporation

Richard Sefton Sidle

Vijayshankar Raman

Ronald Jason Barber

Guy Maring Lohman

Adam J Storm

Chandrasekaran Mohan

Mir Hamid Pirahesh

Sam Sampson Lightstone

Roberta Jo Cochrane

Gopi K Attaluri

Lin Qiao

Pinar Tozun

Bishwaranjan Bhattacharjee

Mohammad Sadoghi Hamedani

Liping Zhang

Xun Xue

Ippokratis Pandis

Steven Robert Pearson

Walter Duke Alvey

Eugene Jon Shekita

Ronen Grosman

Christian Marcelo Garcia-Arellano

Naresh K Chainani

Yuanyuan Tian

David C Sharpe

Rene Mueller

Vincent Kulandaisamy

George Lapis

Min-Soo Kim

Li Zhou

Calisto Paul Zuzarte

Matthew Albert Huras

David Everett Simmen

Tuong Chanh Truong

James John Seeger, Jr

James L Finnie

Roger L Q Zheng

Christina Marie Lee

Monica Sachiye Urata

Antti-Pekka Liedes

Marko Milek

Petrus Kai Chung Chan

Ying-Pong Chen

Viktor Leis

Stefan Arndt

Vijayshanker Raman

Jussi Vuorento

Garret Frederick Swart

Ting Yu Leung

Yi Ding

Frederick Ralph Reiss

F Ryan Johnson

Jae Gil Lee

Serge Philippe Rielau

Wolfgang Lehner

Michael Jeffrey Winer

Ioana Roxana Stanoi

Robbert Van Der Linden

William T O'Connell

Sandeep Tata

Dieu Quang La

Uttam Jain

Chen Luo

James J Seeger

Hamid Pirahesh

Themistoklis Palpanas

Quanzhong Li

Matthew J Spilchen

David C Smith

Tolga Urhan

Viyayshankar Raman

Thomas Fanghaenel

M Hamid Pirahesh

Biao Zhu

Sam B Lightstone

A method includes storing an anchor row vector identification for an anchor row to a local memory. It is determined whether the anchor row vector identification is visible based on isolation requirements. The anchor row vector identification is accessed upon a determination that the anchor row vector identification is visible, and the row vector identification is re-read from the local memory. It is determined whether the anchor row vector identification has not changed since a start of the accessing. Upon a determination that the anchor row vector identification has not changed, read anchor row fields are returned. A first check history is performed on an anchor row history tuple sequence number (TSN) for the anchor row.

In-place updates with concurrent reads in a decomposed state

Embodiments relate to aggregation operations. More specifically, embodiments relate to enforcing an aggregation constraint across a distributed network of compute nodes responding asynchronously. Each compute node individually processes the local partial aggregate value in parallel and asynchronously communicates with a conflict resolution node (CRN).

Scalable enforcement of aggregation constraints within transactions

One embodiment provides for a method including performing, by a processing thread, a process that analyzes transactional operations by maintaining the transactional operations in transaction local side logs, and waiting until a successful transaction commit to append the transaction local side logs to a log stream. The processing thread processes the transactional operations on a key used to determine whether existing data is found for the key. The transactional operations are sped up through parallelism based on partitioning tables across nodes handling the transactional operations. A first process is performed by a first processor that processes updates for values of a key based on updating a first start time table index using unique keys and a start time field of a row for a first appearance of each unique key from the transactional operations.

Resolving versions in an append-only large-scale data store in distributed data management systems

In updating a synopsis table of a database system, a database management unit performs a transaction to insert row(s) in a section of the base table and determines whether a synopsis entry for the section is stored in the memory. If stored in the memory, the in-memory synopsis entry is retrieved and metadata values in the in-memory synopsis entry are updated with data from the row(s) to be inserted. If not stored in the memory, the in-memory synopsis entry is generated and the metadata values in the in-memory synopsis entry are updated with data from the row(s). The insert transaction is then committed. Synopsis entry on-disk updates are thus avoided, significantly reducing the cost of updating the synopsis entries from the insert transaction. This yields enhanced performance especially for inserts of a small number of rows, while the benefits of synopsis entries are still available.

Updating of in-memory synopsis metadata for inserts in database table

According to one or more embodiments of the present invention, a computer-implemented method includes creating a hybrid index for a multi-zone data storage system in which a first zone has data that is organized for transaction processing, and a second zone that has data organized for analytic processing. Each index entry in the hybrid index includes an index key including an equality column, a begin-timestamp column, and a tuple-sequence-number column. The method further includes building an index run to be included in the hybrid index, the index run includes multiple index entries. An index run is created by selecting columns by scanning data blocks from the first zone, computing tuple-sequence-number values for the index entries to be included in the hybrid index, and sorting the index entries according to a predetermined order using the columns to be included in the index run.

Indexing for evolving large-scale datasets in multi-master hybrid transactional and analytical processing systems

A computer-implemented method for processing a transaction may include, upon receiving a transaction change request, operating a coordinator to generate a unique transaction identifier for the transaction request, send a data change request along with the unique transaction identifier to sources, and store the unique transaction identifier along with a respective first change count in a transaction status table. The method may also include, upon receiving a query, operating a given source to determine from the transaction status table, a second change count of sources associated with the unique transaction identifier, determine whether the second change count matches the first change count for each source, and when the first and second change counts match in each of the other sources, read data associated with the query from the given source and each source.

Count and transaction identifier based transaction processing

One embodiment provides for a method for grooming transactional operations in a multi-master distributed data management system including maintaining transactional operations in at least one transaction-local side-log. A table log scan is performed for a table including recording identification for each record in a log. Tuple sequence numbers are generated for the records and the records are written on groomed transaction blocks to the table.

Transactional operations in multi-master distributed data management systems

A method includes performing, by a data structure processor, concurrent read and write operations into a hierarchical data structure that includes a mutable tier including extendible hashing, a hash table, and an immutable tier including a concise hash table (CHT) bitmap. Writers acquire latches on the hierarchical data structure elements that the latches modify. The hierarchical data structure elements are directly accessed by readers without acquiring latches. A modify operation is executed by a writer for one or more levels of the hierarchical data structure. When removed portions of the hierarchical data structure are no longer referenced, tracking is performed by use of a combination of a global state value and a copied local state value.

Concurrent reads and inserts into a data structure without latching or waiting by readers

One embodiment provides for a method including processing transactional operations on a key used to determine whether existing data is found for that key. A first time index is updated using unique keys and a start time field of a first appearance of each key from the transactional operations. A deferred update of prior versions of the key is performed for non-recent data upon determining that recent data in the transactional operations is found for the key.

One embodiment provides a method for efficient representation, access and modification of variable length data objects including converting, by a processor, all object updates into appends. The processor divides each object update into bounded size tokens. The tokens are stored normalized as an anchor and a linked list of fragments, the anchor having a pointer to a linked list of fragments. A fragment comprises a contiguous list of tokens that correspond to the appends.

Efficient representation, access and modification of variable length objects

A method includes performing a lookup using a key into a root node of a multi-tier data structure, to find a partition for performing an insert. A lookup for the key is performed on a first level index that is part of a linked data structure. A payload or reference is added to the linked data structure based on data structure criterion, otherwise the key and the payload are added to the linked data structure if the key is not found. A new first level index is created and added to the linked data structure upon the linked data structure remaining unchanged. The key and the payload or reference are added to the new index. Based on merge criterion, a new second level index is created and a portion of content from selected first level and second level indexes are merged for combining into the new second level index.

Fast multi-tier indexing supporting dynamic update

One embodiment provides a method comprising adjusting a runtime of a dataflow processing environment to operate on multiple batches of objects. The method further comprises pre-allocating one or more vectors of objects, and processing the multiple batches one at a time. The one or more vectors of objects are re-used during processing of each batch.

Efficient dataflow processing for objects

One embodiment provides a method for transparent partial object instantiation for object oriented applications including analyzing, by a processor, application code to determine a list of objects. The processor determines a list of accessed fields for each of the objects based on performing analysis for each function in the application code for accessed fields and other functions invoked.

Partial object instantiation for object oriented applications

A method includes logically organizing, by an object hierarchy processor, data objects in a first hierarchy. A portion of the data objects in the first hierarchy logically includes groupings of other data objects. The object hierarchy processor physically organizes the data objects across two or more types of memory in a second hierarchy. Another portion of the data objects in the second hierarchy physically includes groupings of other data objects. Groupings of the data objects in the second hierarchy are dynamically moved across the two or more types of memory. Levels of access of the data objects are tracked using a data structure that maps groupings of the data objects in the first hierarchy onto metadata information including combined access frequencies of the data objects, and current number of accessors to the data objects, in each grouping of the data objects.

Efficient performance of insert and point query operations in a column store

Embodiments include a method, system, and computer program product for encoding data while it is being processed as part of a query is provided. The method includes receiving a query request and determining a set of values associated with data to be encoded for completing the query request. The method also includes encoding those values such that any subsequent processing operations can be performed on the encoded values to complete the requested query. After performing the subsequent processing operations to complete the requested query, each value is decoded back to its original value.

On-the-fly encoding method for efficient grouping and aggregation

Growing community of inventors

Mountain View, CA, United States of America

Richard Sefton Sidle