Sap Se

Sap Ag

Dirk Thomsen

Ivan Schreter

Thorsten Glebe

Colin Florendo

Blaine French

Axel Schroeder

Werner Thesing

Mihnea Andrei

Tobias Scheuer

Christoph Roterring

David Wein

Christian Bensberg

Till Merker

Yong Sik Kwon

Daniel Booss

Michael Muehle

Martin Heidel

Meinolf Block

Sascha Zorn

Beomsoo Kim

Reiner Singer

Myung Sun Park

Martin Schindewolf

Patrick Voelker

Christoph Hohner

Johannes Beigel

Martin Kittel

Holger Mack

Fabian Garagnon

Sergej Hardock

Andreas Hartel

Blaise Lengrand

Ivan Schre'ter

Johannes Gloeckle

Computer-readable media, methods, and systems are disclosed for improving performance when dropping database snapshots by linking converter streams associated with adjacent snapshots in a database system. A request is received to drop a snapshot. The database system iterates a first converter tree associated with a first converter stream to identify a first physical page number to be loaded. A physical page corresponding to the first physical page number is loaded. The database system iterates a second converter tree associated with a second converter stream to identify a second physical page number to be loaded. The second physical page number is queried from an iterator associated with the first converter tree. Responsive to determining that the physical page associated with the second physical page number is not present in main memory, the in-memory database system loads from disk, the physical page associated with the second physical page number.

Performant dropping of snapshots by linking converter streams

Embodiments of the present disclosure include techniques for backing up data. In one embodiment, a plurality of read requests are issued. In response to the read requests, a plurality of data pages are retrieved. The plurality of data pages are stored in a plurality of buffers. During said storing, for each data page, an indication that storage of a particular data page of the plurality of data pages has been completed is generated. In response to an indication that storage of a particular data page has been completed, the data page is processed with one of a plurality of jobs, where a plurality of data pages are processed by the plurality of jobs in parallel.

Systems and methods for backing up data

Computer-readable media, methods, and systems are disclosed for processing backup and recovery of pages in an in-memory database system employing tenant-based, group-level encryption for a plurality of tenants. Page metadata for a page may be collected. The page may comprise a header and a set of page contents. The page metadata may be stored with the page header. When a backup request is received, the page may be loaded into a backup stream. The page may be loaded with the header unencrypted and the page contents encrypted. When a recovery request is received, the data page may be retrieved from the backup stream. A converter may be rebuilt using the page metadata. The data page may then be written to the main memory of the database system. The page contents may be decrypted by the converter.

Backup and recovery under group-level encryption

A database system includes a persistent storage system, a memory storing metadata defining a tenant object and a plurality of database artifacts, a first instance of the tenant object, the first instance associated with a first plurality of the database artifacts including first data associated with the first instance of the tenant object, and a second instance of the tenant object, the second instance associated with a second plurality of the database artifacts including second data associated with the second instance of the tenant object. A processing unit is to execute program code of a database instance to cause the database system to encrypt the first data associated with the first instance of the tenant object using a first public encryption key and store the encrypted first data in the persistent storage system, and encrypt the second data associated with the second instance of the tenant object using a second public encryption key and store the encrypted second data in the persistent storage system.

Native multi-tenant encryption for database system

Computer-readable media, methods, and systems are disclosed for tenant-specific encryption of container in connection with a database employing group-level encryption. An encryption group identifier may be assigned to container. The encryption group identifier may define how the container is encrypted. A container entry corresponding to the container may be created. A commit operation may be received for committing the assignment of the encryption group identifier to the container. A job may be initialized for encryption the container according to the encryption group identifier. The container may be flagged as modified. A flush operation may be initiated whereby the container is re-encrypted according to the encryption group identifier. Once flushing is complete, the container entry may be deleted.

Handling pre-existing containers under group-level encryption

Computer-readable media, methods, and systems are disclosed for handling intermediate data in connection with a database employing group-level encryption. Intermediate data is used during database operation and stored transiently such that the intermediate data is removed from memory upon database restart. To protect the privacy of the intermediate data, a random encryption key may be generated upon startup of a database instance. The random encryption key may be stored transiently. During database operation, the random encryption key may be used to encrypt and/or decrypt the intermediate data. The transient memory may be wiped upon database shut down such that the random encryption key is no longer accessible upon database restart.

Encrypting intermediate data under group-level encryption

Computer-readable media, methods, and systems are disclosed for minimizing data volume growth in a database system under changes to an encryption status of a plurality of data pages persisted to a database. Initially, a request is received to update an encryption parameter associated with the database. Next, it is determined whether a candidate page requires encryption changes. In response to determining that the candidate page is not currently in use by one or more active database snapshots and not currently loaded in main memory, the candidate page is loaded into main memory. Next, an encryption operation is performed on the candidate page, and the encrypted page is designated for persistence. Finally, based on a current number of candidate pages already encrypted during a current save point cycle, the selective iteration is paused until a subsequent save point cycle.

Minimizing data volume growth under encryption changes

Computer-readable media, methods, and systems are disclosed for encrypting and decrypting data pages in connection with a database employing group-level encryption. A request to load a group-level encrypted logical data page into main memory is received, the data page being identified by a logical page number. A block of group-level encrypted data is loaded into the main memory of the database system from an address corresponding to the physical block number. A block of group-level encrypted data is loaded into the main memory of the database system. A header associated with the block of group-level encrypted data is decrypted using a data-volume encryption key, and an encryption-group identifier is accessed from the decrypted header. A group-level encryption key is retrieved from a key manager, and the remainder of the block of group-level encrypted data is decrypted using the group-level encryption key.

Processing data pages under group-level encryption

Computer-readable media, methods, and systems are disclosed for processing log entries in an in-memory database system employing tenant-based, group-level encryption for a plurality of tenants. A request to generate a database transaction log record is received. A log entry handle corresponding to the allocated log buffer is provided. In response to determining that the transaction log record to be written into the log buffer contains tenant-specific content, certain content requiring group-level encryption is flagged. An encryption group identifier is received, and the tenant-specific content is encrypted with a corresponding group-level encryption key. The group-level encryption group identifier is appended to the transaction log header, and log data containing the log buffer is encrypted with one or more encryption keys. Finally, the encrypted log data is persisted and subsequently read, unencrypted, and replayed under appropriate circumstances.

Processing log entries under group-level encryption

Computer-readable media, methods, and systems are disclosed for managing group-level database encryption keys under group-level encryption in a database management system. Upon startup of the database management system, persisted database entries are sequentially processed to produce an in-memory data structure comprising a set of encryption group identifier metadata tuples having an encryption group identifier and a valid-from save point cycle version. The set of encryption group identifier metadata tuples is mapped to a set of key identifier tuples including a local secure store identifier and a group-level encryption key identifier. A set of group-level encryption keys is received from a key management system, according to which a group-level encryption key is mapped to each encryption group identifier metadata tuple. Finally, an in-memory representation of the mapping between the set of encryption group identifier metadata tuples, the set of key identifier tuples, and the set of group-level encryption keys is constructed.

Managing encryption keys under group-level encryption

Computer-readable media, methods, and systems are disclosed for improving performance when dropping database snapshots in an in-memory database by pruning tree branches in one or more converters associated with a snapshot of an in-memory database. A request to compare at least two database snapshots is received. Physical pages associated with at least one of the two database snapshots are loaded. First and second converter streams are iterated. Physical block numbers of intermediate converter pages to which the first and second converter streams currently point are compared. If the physical page numbers are all non-equal, the physical pages referenced by the second converter stream are set to free. If any of the physical page numbers are equal, subsequent traversal of a corresponding converter subtree in the first and second converter streams may be skipped to improve performance.

Performant dropping of snapshots by converter branch pruning

Recovery of an in-memory database is initiated. Thereafter, pages for recovery having a size equal to or below a pre-defined threshold are copied to a superblock. For each copied page, encryption information is added to a superblock control block for the superblock. The copied pages are encrypted within the superblock using the corresponding encryption information added to the super block control block. The superblock is then flushed from memory (e.g., main memory, etc.) of the database to physical persistence.

Small database page recovery

A provisional page to be filled with data is allocated in an in-memory database system in which pages are loaded into memory and having associated physical disk storage a provisional page to be filled with data. Thereafter, the provisional page is filled with data. The provisional page is register after the provisional page has been filled with data such that consistent changes in the database are not required for the provisional page prior to the registering.

In-memory database page allocation

A database receives pages that are piped from backup media with each of the pages having a corresponding savepoint version. At least a portion of the pages are then flagged as being from recovery. The savepoint versions for the pages are maintained if they have a flag. A single checksum is then calculated for each page to confirm integrity. Each page is then loaded into memory of the database after it is confirming the corresponding calculated single checksum. Related apparatus, systems, techniques and articles are also described.

Growing community of inventors

Berlin, Germany

Dirk Thomsen