International Business Machines Corporation

Kyungpook National University Industry-academic Cooperation Foundation

Guy Maring Lohman

Vijayshankar Raman

Ronald Jason Barber

Richard Sefton Sidle

Ippokratis Pandis

Sam Sampson Lightstone

David Everett Simmen

Adam J Storm

Eugene Jon Shekita

Peter Haas

Lin Qiao

Jun Rao

Rene Mueller

Mir Hamid Pirahesh

Bishwaranjan Bhattacharjee

Chandrasekaran Mohan

Mohammad Sadoghi Hamedani

Tanveer Fathima Syeda-Mahmood

Gopi K Attaluri

John Michael Lake

Xun Xue

Liping Zhang

Yinan Li

Nimrod Megiddo

Monica Sachiye Urata

Naresh K Chainani

Volker Gerhard Markl

Vincent Kulandaisamy

Min-Soo Kim

Calisto Paul Zuzarte

Bruce Gilbert Lindsay

Ting Yu Leung

Eileen Tien Lin

Chun Zhang

Pinar Tozun

Wayne Jonleon Young

Gregory R Stager

Stefan Arndt

Viktor Leis

Ian R Finlay

Kevin Scott Beyer

Mark Francis Wilding

Christian Marcelo Garcia-Arellano

Ronen Grosman

Yuanyuan Tian

Robert S Germain

Daniele Costante Zilio

Gary Valentin

Daniel Costante Zilio

Sandeep Tata

Markos Zaharioudakis

Dengfeng Gao

Ihab F Ilyas

Jae Gil Lee

Quanzhong Li

Marki Volker

Debabrata Dash

Vincent Kulandai Samy

Wei Fan

Vanja Josifovski

Berthold Reinwald

Roberta Jo Cochrane

Rajeev Gupta

Michael Jeffrey Winer

Stefan Bengt Edlund

Walid Rjaibi

Ioana Roxana Stanoi

John Davis Palmer

David C Sharpe

Natwar Modani

Tanya Couch

Laurent Sebastien Mignet

Latha S Colby

Robbert Van Der Linden

William T O'Connell

George Lapis

Holger Kache

Ivan Popivanov

Minglong Shao

Bernhard K Schiefer

Ashutosh Singh

Uttam Jain

Hamid Pirahesh

Tim Kaldewey

Stephan Eberhard Ewen

Kiyoshi Ono

Tanveer Fathima Mahmood

Julia Stoyanovich

Robin D Van Boeschoten

Wook-Shin Han

Wooseong Kwak

Jinsoo Lee

Volker Markl

Kit Man Cheung

Samuel S Lightstone

Ashutosh Vir Vikram Singh

Michael Stillger

Johann Christoph Freytag

Peter B Volk

Viyayshankar Raman

A method includes storing an anchor row vector identification for an anchor row to a local memory. It is determined whether the anchor row vector identification is visible based on isolation requirements. The anchor row vector identification is accessed upon a determination that the anchor row vector identification is visible, and the row vector identification is re-read from the local memory. It is determined whether the anchor row vector identification has not changed since a start of the accessing. Upon a determination that the anchor row vector identification has not changed, read anchor row fields are returned. A first check history is performed on an anchor row history tuple sequence number (TSN) for the anchor row.

In-place updates with concurrent reads in a decomposed state

One embodiment provides for a method including performing, by a processing thread, a process that analyzes transactional operations by maintaining the transactional operations in transaction local side logs, and waiting until a successful transaction commit to append the transaction local side logs to a log stream. The processing thread processes the transactional operations on a key used to determine whether existing data is found for the key. The transactional operations are sped up through parallelism based on partitioning tables across nodes handling the transactional operations. A first process is performed by a first processor that processes updates for values of a key based on updating a first start time table index using unique keys and a start time field of a row for a first appearance of each unique key from the transactional operations.

Resolving versions in an append-only large-scale data store in distributed data management systems

Embodiments of the invention relate to processing data records, and for a multi-phase partitioned data reduction. The first phase relates to processing data records and partitioning the records into a first partition of records having a common characteristic and a second partition of records that are not members of the first partition. The data records in each partition are subject to intra-partition data reduction responsive to a resource constraint. The data records in each partition are also subject to an inter-partition data reduction, also referred to as an aggregation to reduce a footprint for storing the records. Partitions and/or individual records are logically aggregated and a data reduction operation for the logical aggregation of records takes place in response to available resources.

Accurate partition sizing for memory efficient reduction operations

A method includes performing, by a data structure processor, concurrent read and write operations into a hierarchical data structure that includes a mutable tier including extendible hashing, a hash table, and an immutable tier including a concise hash table (CHT) bitmap. Writers acquire latches on the hierarchical data structure elements that the latches modify. The hierarchical data structure elements are directly accessed by readers without acquiring latches. A modify operation is executed by a writer for one or more levels of the hierarchical data structure. When removed portions of the hierarchical data structure are no longer referenced, tracking is performed by use of a combination of a global state value and a copied local state value.

Concurrent reads and inserts into a data structure without latching or waiting by readers

One embodiment provides for a method including processing transactional operations on a key used to determine whether existing data is found for that key. A first time index is updated using unique keys and a start time field of a first appearance of each key from the transactional operations. A deferred update of prior versions of the key is performed for non-recent data upon determining that recent data in the transactional operations is found for the key.

One embodiment provides a method for efficient representation, access and modification of variable length data objects including converting, by a processor, all object updates into appends. The processor divides each object update into bounded size tokens. The tokens are stored normalized as an anchor and a linked list of fragments, the anchor having a pointer to a linked list of fragments. A fragment comprises a contiguous list of tokens that correspond to the appends.

Efficient representation, access and modification of variable length objects

A method includes performing a lookup using a key into a root node of a multi-tier data structure, to find a partition for performing an insert. A lookup for the key is performed on a first level index that is part of a linked data structure. A payload or reference is added to the linked data structure based on data structure criterion, otherwise the key and the payload are added to the linked data structure if the key is not found. A new first level index is created and added to the linked data structure upon the linked data structure remaining unchanged. The key and the payload or reference are added to the new index. Based on merge criterion, a new second level index is created and a portion of content from selected first level and second level indexes are merged for combining into the new second level index.

Fast multi-tier indexing supporting dynamic update

A method of orchestrated shuffling of data in a non-uniform memory access device including a plurality of processing nodes that connected by interconnects. The method includes running an application on a plurality of threads executing on the plurality of processing nodes. Data to be shuffled is identified from source threads running on source processing nodes among the processing nodes to target threads executing on target processing nodes among the processing nodes. The method further includes generating a plan for orchestrating the shuffling of the data among the all of the memory devices associated with the threads and for simultaneously transmitting data over different interconnects to a plurality of different target processing nodes from a plurality of different source processing nodes. The data is shuffled among all of the memory devices based on the plan and each processing node is capable of accessing data from first and second local memory devices.

Data shuffling in a non-uniform memory access device

A method includes logically organizing, by an object hierarchy processor, data objects in a first hierarchy. A portion of the data objects in the first hierarchy logically includes groupings of other data objects. The object hierarchy processor physically organizes the data objects across two or more types of memory in a second hierarchy. Another portion of the data objects in the second hierarchy physically includes groupings of other data objects. Groupings of the data objects in the second hierarchy are dynamically moved across the two or more types of memory. Levels of access of the data objects are tracked using a data structure that maps groupings of the data objects in the first hierarchy onto metadata information including combined access frequencies of the data objects, and current number of accessors to the data objects, in each grouping of the data objects.

Efficient performance of insert and point query operations in a column store

Embodiments include a method, system, and computer program product for encoding data while it is being processed as part of a query is provided. The method includes receiving a query request and determining a set of values associated with data to be encoded for completing the query request. The method also includes encoding those values such that any subsequent processing operations can be performed on the encoded values to complete the requested query. After performing the subsequent processing operations to complete the requested query, each value is decoded back to its original value.

On-the-fly encoding method for efficient grouping and aggregation

A method of orchestrated shuffling of data in a non-uniform memory access device that includes a plurality of processing nodes that are connected by interconnects. The method includes running an application on a plurality of threads executing on the plurality of processing nodes. Data to be shuffled is identified from source threads running on source processing nodes among the processing nodes to target threads executing on target processing nodes among the processing nodes. The method further includes generating a plan for orchestrating the shuffling of the data among the all of the memory devices associated with the threads and for simultaneously transmitting data over different interconnects to a plurality of different target processing nodes from a plurality of different source processing nodes. The data is shuffled among all of the memory devices based on the plan. The data includes operand data and operational state data of the source threads.

A method of orchestrated shuffling of data in a non-uniform memory access device that includes a plurality of processing nodes connected by interconnects. The method includes running an application on a plurality of threads executing on the plurality of processing nodes. Data to be shuffled is identified from source threads running on source processing nodes among the processing nodes to target threads executing on target processing nodes among the processing nodes. The method further includes generating a plan for orchestrating the shuffling of the data among the all of the memory devices associated with the threads and for simultaneously transmitting data over different interconnects to a plurality of different target processing nodes from a plurality of different source processing nodes. The data is shuffled among all of the memory devices based on the plan. Shifting the data-shifting table includes rotating a first ring with respect to a second ring.

A method includes setting, by an update processor, a write latch in a first data structure associated with an object. The first data structure is copied to a storage structure. A history tuple sequence number (TSN) of the first data structure is set to point to a TSN of the copied first data structure. The version identifier is set to point to a transaction identification for the object. Data portions are updated for the first data structure. The version identifier is read from the first data structure. It is determined whether the version identifier of the first data structure is visible for a transaction including isolation requirements. If version identifier of the first data structure is visible, the first data structure is accessed and it is determined whether the version identifier of the first data structure changed since starting the transaction.

A database table is partitioned with column-major layout by limiting partitioning one or more columns containing join-column values for a join operator. Join operations are executed for joining the partitioned columns.

Efficient partitioned joins in a database with column-major layout

A method includes joining data between at least two data sets. Values of one or more join attributes of each of the data sets is represented in a compressed form, indicated by an encoding scheme. A compression scheme for the one or more join attributes is dynamically selected.

Efficient join on dynamically compressed inner for improved fit into cache hierarchy

Growing community of inventors

San Jose, CA, United States of America

Guy Maring Lohman