International Business Machines Corporation

Google Inc.

Eugene Jon Shekita

Guy Maring Lohman

Marcus Felipe Fontoura

Sridhar Rajagopalan

Bruce Gilbert Lindsay

David Everett Simmen

Andreas Neumann

Vijayshankar Raman

Ronald Jason Barber

Lin Qiao

John A McPherson, Jr

Kevin Scott Beyer

Runping Qi

Mir Hamid Pirahesh

Richard Sefton Sidle

Ronny Lempel

Andrei Zary Broder

Ning Li

Jason Yeong Zien

Nadav Eiron

Monica Sachiye Urata

Jayavel Shanmugasundaram

Min-Soo Kim

Gopi K Attaluri

Peter Haas

Vuk Ercegovac

Michael Herscovici

Fei Xu

Stefan Arndt

Reiner Kraft

Sam Sampson Lightstone

Roger C Raphael

Ting Yu Leung

Honesty Cheng Young

Jun Rao

Harish Deshmukh

Shila Ofek-Koifman

Felipe Marcus Fontoura

Robert William Lyle

Timothy Ray Malkemus

Gerald George Kiernan

Paul Sherwood Taylor

Fatma Ozcan

Jyh-Herng Chow

Rajasekar Krishnamurthy

Beverly Yang

Alexander Johannes Smola

John Frederick Hornibrook

Tony Leung

Michael James Carey

Latha S Colby

Tony Kai-chi Leung

Sandeep Tata

Amit Radheshyam Somani

Dennis Craig Fetterly

Mu Li

Kaarel Truuvert

Amr Ahmed

Bor-Yiing Su

Quanzhong Li

Geoffrey Peddle

Surendra Kumar Verma

Zografoula Vagena

Steffen Rendle

Bor-yiing Su

Narayana Iyer

Systems and associated methods for highly parallel processing of parameterized simulations are described. Embodiments permit processing of stochastic data-intensive simulations in a highly parallel fashion in order to distribute the intensive workload. Embodiments utilize methods of seeding records in a database with a source of pseudo-random numbers, such as a compressed seed for a pseudo-random number generator, such that seeded records may be processed independently in a highly parallel fashion. Thus, embodiments provide systems and associated methods facilitating quicker data-intensive simulation by enabling highly parallel asynchronous simulations.

Systems and methods for highly parallel processing of parameterized simulations

The present disclosure provides a new scalable coordinate descent (SCD) algorithm and associated system for generalized linear models whose convergence behavior is always the same, regardless of how much SCD is scaled out and regardless of the computing environment. This makes SCD highly robust and enables it to scale to massive datasets on low-cost commodity servers. According to one aspect, by using a natural partitioning of parameters into blocks, updates can be performed in parallel a block at a time without compromising convergence. Experimental results on a real advertising dataset are used to demonstrate SCD's cost effectiveness and scalability.

Robust large-scale machine learning in the cloud

A database table is partitioned with column-major layout by limiting partitioning one or more columns containing join-column values for a join operator. Join operations are executed for joining the partitioned columns.

Efficient partitioned joins in a database with column-major layout

Embodiments of the present invention provide a database processing system for efficient partitioning of a database table with column-major layout for executing one or more join operations. One embodiment comprises a method for partitioning a database table with column-major layout, partitioning only the join-columns by limiting the partitions by size and number, executing one or more join operations for joining the partitioned columns, and optionally de-partitioning the join result to the original order by sequentially writing and randomly reading table values using P cursors.

System, method and computer program products for probing a hash table by receiving a compressed input key, computing a hash value for the compressed input key and probing one or more buckets in a hash table for a match. Each bucket includes multiple chunks. For a bucket in the hash table, chunks are searched in that bucket by comparing in parallel the hash value with multiple slots in each chunk, such that if a value in a chunk equals the hash value of the compressed input key, then a match is declared and a vector is returned with a significant bit of a matching slot in the bucket set to a value. If a value stored in a chunk corresponds to an empty slot, then a mismatch is declared, and the vector is returned as the result with the significant bit of a matching empty slot set to the value.

Systems, methods and computer program products for probing a hash table for improved latency and scalability in a processing system

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for handling faults in a distributed key-value storage system. One of the methods includes receiving an indication that a machine storing a primary replica of a first replication chain is inactive, in response to receiving the indication, generating a concatenated replica comprising a first replica of the first replication chain and a second replica of a second replication chain, the second replication chain comprising replicas of a second key segment, the second key segment being adjacent to the first key segment in the multiple key segments of the plurality of keys, and providing, to another machine in the ordered sequence of machines, a notification of availability of the concatenated replica.

Fault tolerant distributed key-value storage

A method for storing database information includes storing a table having data values in a column major order. The data values are stored in a list of blocks. The method also includes assigning a tuple sequence number (TSN) to each data value in each column of the table according to a sequence order in the table. The data values that correspond to each other across a plurality of columns of the table have equivalent TSNs. The method also includes assigning each data value to a partition based on a representation of the data value. The method also includes assigning a tuple map value to each data value. The tuple map value identifies the partition in which each data value is located.

Intra-block partitioning for database management

System, method and computer program products for storing data by computing a plurality of hash functions of data values in a data item, and determining a corresponding memory location for one of the plurality of hash functions of data values in the data item. Each memory location is of a cacheline size wherein a data item is stored in a memory location. Each memory location can store a plurality of data items. A key portion of all data items is contiguously stored within the memory location, and a payload portion is contiguously stored within the memory location. Payload portions are packed as bit-aligned in a fixed-sized memory location, comprising a bucket in a bucketized hash table, each bucket sized to store multiple key portions and payload portions that are packed as bit-aligned in a fixed-sized bucket. Corresponding key portions are stored as compressed keys in said fixed-sized bucket.

Systems, methods and computer program products for reducing hash table working-set size for improved latency and scalability in a processing system

A method for updating a scalable row-store, including: receiving an update to a key within a range of keys in a database table, wherein the database table is distributed across nodes in a cluster of computing devices; and replicating the update over a group of the nodes using a consensus-based replication algorithm, wherein the replication algorithm includes completing the update in response to receiving acknowledgement messages from a majority of the nodes in the group indicating that the majority has received notification of the update.

Scalable row-store with consensus-based replication

Techniques are disclosed for synchronizing a primary data system with an auxiliary data system that processes data for the primary data system. In one embodiment, how current the primary data system and the auxiliary data system are is determined. Requests sent from the primary data system that were not processed by the auxiliary data system are determined. The requests are resent to the auxiliary data system for processing.

Synchronizing an auxiliary data system with a primary data system

Provided are techniques for partitioning a physical index into one or more physical partitions; assigning each of the one or more physical partitions to a node in a cluster of nodes; for each received document, assigning an assigned-doc-ID comprising an integer document identifier; and, in response to assigning the assigned-doc-ID to a document, determining a cut-off of assignment of new documents to a current virtual-index-epoch comprising a first set of physical partitions and placing the new documents into a new virtual-index-epoch comprising a second set of physical partitions by inserting each new document to a specific one of the physical partitions in the second set using one or more functions that direct the placement based on one of the assigned-doc-id, a field value derived from a set of fields obtained from the document, and a combination of the assigned-doc-id and the field value.

Index partition maintenance over monotonically addressed document sequences

Growing community of inventors

San Jose, CA, United States of America

Eugene Jon Shekita