International Business Machines Corporation

Ibm Canada Limited - Ibm Canada Limitee

David C Kalmuk

Paul Miller Bird

Scott Douglas Walkty

Hebert Walter Pereyra

Xun Xue

Jon A Lind

Patrick R Perez

Calisto Paul Zuzarte

Sam Sampson Lightstone

Ian R Finlay

Christian Marcelo Garcia-Arellano

Timothy Ray Malkemus

Naresh K Chainani

Torsten Steinbach

Kiran Chinta

Gordon Andrew Booman

Hamdi Roumani

Shaorong Liu

Stephen Rees

Xing Yi Chen

Jack Hon Wai Ng

Paul C Huffman

Herbert Walter Pereyra

Alexander Cheung

Faizan Qazi

Hebert W Pereyra

Ronald Jason Barber

Matthew Albert Huras

Mark Francis Wilding

Shaikh Shahriar Quader

Marin Litoiu

Piotr Mierzejewski

John Paul Kennedy

Emmanouil Papangelis

Robert C Hooper

Richard Sefton Sidle

Ketan Rampurkar

Cheuk Lun Lam

Nicolas Andres Jaramillo Duran

Sumona Mukhopadhyay

Matthew A Huras

Mark F Wilding

John P Kennedy

Examples described herein provide a computer-implemented method for storing data pages of a database in a remote storage. The method includes receiving a write request associated with one or more data pages of the database. The method further includes storing, in a first storage layer having a write ahead log, the write request associated with the one or more data pages. The method further includes storing, in a first write buffer, the received write request associated with the one or more data pages that each include a legacy data page identifier. The method further includes storing, in a second storage layer having a second write buffer, a sorted string table (SST) of the received write request. The method further includes storing, in a top level of a log structured merge (LSM) tree disposed in the remote storage, the SST from the second write buffer.

Storing data pages of a database in a cloud object storage

Described are techniques for creating a point-in-time backup of application data stored on disparate storage systems. The techniques include initiating a snapshot of the first datastore. The techniques further include initiating a suspension of delete operations at the second datastore to preserve immutable application data stored on the second datastore. The techniques further include initiating a background copy of the second datastore. The techniques further include initiating a resumption of the delete operations at the second datastore in response to completion of the background copy of the second datastore.

Point-in-time backup of application data stored on disparate storage systems

A system and method is provided for dynamic renewable runtime resource management in response to flexible resource allocations by a processor. In embodiments, a method includes: calculating, by a processor of a system, a resource consumption value of a first workload by aggregating allocation values of persistent resources currently allocated to the first workload by the processor; determining, by the processor, that the resource consumption value of the first workload is greater than a predefined resource allocation target for the first workload; and temporarily adjusting, by the processor, a renewable runtime resource target of the first workload from an initial target value to a temporary target value based on the resource consumption value.

Dynamic renewable runtime resource management

A DBMS training subsystem trains a DBMS workload-manager model with training data identifying resources used to execute previous DBMS data-access requests. The subsystem integrates each request's high-level features and compile-time operations into a vector and clusters similar vectors into templates. The requests are divided into workloads each represented by a training histogram that describes the distribution of templates associated with the workload and identifies the total amounts and types of resources consumed when executing the entire workload. The resulting knowledge is used to train the model to predict production resource requirements by: i) organizing production queries into candidate workloads; ii) deriving for each candidate a histogram similar in form and function to the training histograms; iii) using the newly derived histograms to predict each candidate's resource requirements; iv) selecting the candidate with the greatest resource requirements capable of being satisfied with available resources; and v) executing the selected workload.

Learning-based workload resource optimization for database management systems

A computer-implemented method for a partitioned bloom filter merge is provided. A non-limiting example of the computer-implemented method includes partitioning, by a processing device, a bloom filter into N equal size filter partitions. The method further includes distributing, by the processing device, each of the filter partitions to an associated node. The method further includes merging, by the processing device, the filter partitions in each of the associated nodes. The method further includes redistributing, by the processing device, the merged filter partitions to each of the N nodes. The method further includes joining, by the processing device, the merged filter partitions in each of the N nodes to assemble a complete merged bloom filter.

Partitioned bloom filter merge for massively parallel processing clustered data management

Predictive job admission control is provided. In response receiving a job for execution, a predicted resource utilization estimate is generated for the job prior to admission of the job to execution. Historic job execution statistics corresponding to the job are searched for. It is determined whether corresponding historic job execution statistics were found during the search. In response to determining that corresponding historic job execution statistics were found during the search, the predicted resource utilization estimate for the job is adjusted based on the corresponding historic job execution statistics found during the search to form an adjusted resource utilization estimate. The job is scheduled for execution based on the adjusted resource utilization estimate.

Predictive job admission control with feedback

A computer-implemented method includes receiving a request to transmit column group data to a target node, the column group data comprising C columns within a column-oriented data table and determining a transmission row count R for transmitting the column group data to the target node. The method may also include transmitting a transmission packet comprising R sequentially-ordered data elements for each of the C columns to the target node. The R data elements for each column may be sequentially retrieved from memory. A corresponding method includes receiving, at a target node, a request to receive the column group data, determining the transmission row count R for receiving the column group data, and receiving a transmission packet comprising R sequentially-ordered data elements for each of the C columns and storing the sequentially-ordered data elements within memory. A corresponding computer system and corresponding computer program products are also disclosed herein.

Data transfers in columnar data systems

A computer-implemented method for scheduling a set of jobs executed in a computer system can include determining a workload-time parameter for a set of at least one job. The workload-time parameter can relate to execution-time parameters for the set of at least one job. The method can include determining a schedule tuning parameter for the set of at least one job, the schedule tuning parameter based on the workload-time parameter. The method can include generating a scheduling factor for each job in the set, the scheduling factor generated based on the schedule tuning parameter. The method can include scheduling the set of at least one job based on the scheduling factor.

Tunable computerized job scheduling

A middleware processor provisioning process provisions a plurality of processors in a multi-processor environment. The processors themselves may be subdivided in to one or more partitions or processing instances for which a single processing queue is created and a single kernel thread is started. User processing requests are portioned and dispatched across the plurality of processing queues and are serviced by the corresponding kernel process, thereby efficiently using available processing resources while servicing the user processing requests in a desired manner.

Processor provisioning by a middleware processing system

A middleware processor provisioning process provisions a plurality of processors in a multi-processor environment. The processing capability of the multiprocessor environment is subdivided and multiple instances of service applications start protected processes to service a plurality of user processing requests, where the number of protected processes may exceed the number of processors. A single processing queue is created for each processor. User processing requests are portioned and dispatched across the plurality of processing queues and are serviced by protected processes from corresponding service applications, thereby efficiently using available processing resources while servicing the user processing requests in a desired manner.

Processor provisioning by a middleware processing system for a plurality of logical processor partitions

According to an embodiment of the present invention, a system assigns at least one workload a hard share quantity and at least one other workload a soft share quantity or a hard share quantity. The system allocates a resource to the workloads based on the hard share quantity and the soft share quantity of active workloads in a predefined interval. A hard share quantity indicates a maximum resource allocation and a soft share quantity enables allocation of additional available processor time. Embodiments of the present invention further include a method and computer program product for allocating a resource to workloads in substantially the same manner as described above.

Adaptive resource usage limits for workload management

Growing community of inventors

Toronto, Canada

David C Kalmuk