Emc IP Holding Company LLC

Amazon Technologies, Inc.

Hemanth Satyanarayana

Abhinav Duggal

Tony T Wong

Punit Rajgaria

Murali Brahmadesam

Ramesh Chander

Gokul Soundararajan

Ippokratis Pandis

Gaurav Saxena

Sanuj Basu

Matthew Perry Abrams

Gopal Paliwal

Todd Jeffrey Green

Vadim Skipin

Michael Higgins Dowling

Omar Farhat

Aakash Ashwin Shah

Johannes Wust

Ranganathan Dhathri Purohith

Techniques for fast database recovery in a multi-volume database environment via transactional awareness are described. In the event of a failure associated with a first volume storing database page data, the first volume can be restored to a point in time and transactional metadata from a second volume storing logical change data can be obtained for a limited number of transactions occurring at/after that point in time, as opposed to analyzing extremely large change log files. These transactions can be checked to ensure that they have all been persisted, and if not, change data for those transactions can be obtained from the second volume and used to replay these transactions on the restored first volume.

Fast database recovery in a multi-volume database environment via transactional awareness

Techniques for volume durable log sequence number movement in a multi-volume database environment are described. Blocks of database updates are persisted in an atomic and durable manner, where the blocks may include updates to a first volume, a second volume, or both. Backlinks from one block of updates to the previous block of updates may be set, in addition to backlinks between updates to the first volume and backlinks between updates to the second volume. Upon durably persisting a block, the block backlinks can be followed to help verify that all contiguous blocks—of changes to one or both volumes—have all been persisted. Thereafter, a volume durable log sequence number (VDL) pointer value can be updated to point to a known 'last' durably persisted log update.

Volume durable log sequence number movement in a multi-volume database environment

Methods for replicating transactional tables of a transactional database to an analytical database and maintaining updates to those transactional table representations are disclosed. Snapshots of the transactional tables are provided to the analytical database via a transport mechanism, such as a data storage service or a data streaming service, and stored at the analytical database. Then, checkpoints comprising portions of a change-data-capture log that has recorded transactional changes to the transactional tables of the transactional database are provided to the analytical database via the same or different transport mechanism and used to commit those transactional changes to the snapshot representations. The snapshot representations may be used to respond to incoming analytical queries in order to provide real-time querying results.

Hybrid transactional and analytical processing architecture for optimization of real-time analytical querying

Embodiments for allocating and reclaiming memory using dynamic buffer allocation for a slab memory allocator. The method keeps track of a count of a total number of worker threads and a count of a total number of quiesced threads, and determines if there is any free slab memory. If there is no free slab memory, the method triggers an out of memory event and increments the count of the total number of quiesced threads. It reclaims all objects currently allocated in an object pool, and allocates a buffer of a next smaller size than an original buffer until a sufficient amount of slab memory is freed.

Slab memory allocator with dynamic buffer resizing

Embodiments for a mostly unique file selection process for a deduplication backup system are described. The process assigns tags to files. A tag serves as a hint about the similarity of files in a deduplication file system. It is expected that files from the same client machine will be assigned the same tag. The tag is the smallest unit of migration and serves as a hint of the similarity of the files. The MUFS process measures the uniqueness using a u-index that is a function of the total unique size of a tag relative to the total size of the tag. A load balancer then selects the most unique tags for migration to free the maximum space. It uses the u-index to measure the uniqueness percentage of a tag, so that tags with the highest u-index are selected for migration to free up maximum space on the source node.

Mostly unique file selection method for deduplication backup systems

Embodiments for a memory efficient perfect hashing for large records. A container ID set is divided into multiple fixed range sizes. These ranges are then mapped into perfect hash buckets until each bucket is filled to uniformly distribute the container IDs across different perfect hash buckets so that the number of CIDs in every perfect hash bucket is the same or nearly the same. Individual perfect hash functions are created for each perfect hash bucket. With container IDs as keys, the process maps n keys to n positions to reduce any extra memory overhead. The perfect hash function is implemented using a compress, hash, displace (CHD) algorithm using two levels of hash functions. The level 1 hash functions divides the keys into multiple internal buckets with a defined average number of keys per bucket. The CHD algorithm iteratively tries different level 2 hash variables to achieve collision-free mapping.

Memory efficient perfect hashing for large records

Embodiments for dynamically resizing buffers for a slab allocator process are described. The slab allocator informs the consumer that the memory buffer must be shrunk to a smaller size. A buffer allocation process dynamically reclaims portions of larger memory buffers to make room for a smaller allocation by shrinking data objects in larger slabs and returning slabs to reserve or free slab lists. Initially a large limit is set, and it is dynamically reduced once all the available memory is exhausted. This allows the slab allocator to adapt to the workload.

Growing community of inventors

Santa Clara, CA, United States of America

Hemanth Satyanarayana