Amazon Technologies, Inc.

Yahoo! Inc.

Sandeep Kumar

Chakravarthi Kalyana Valicherla

Varun Verma

Danny Wei

Lalit Jain

Shailendra Verma

Marc Stephen Olson

Ankit Singh

Venkatesh Nagapudi

Anil Gathala

Arvind Chandrasekar

Gregory Branchek Roth

Gregory Alan Rubin

Mark Christopher Seigle

Sriram Venugopal

Oscar Allen Grim Courchaine

Kamran Tirdad

Ashish Palekar

Bryan James Donlan

James T Pinkerton

Shobha Agrawal

Kristina Kraemer Brenneman

Rajiv Puranik

Rucha Nene

Anirudha Singh Bhadoriya

Timothy Ross McCune

Shlomo Yehezkel

Jay Ramamurthi

ChyrSong Ting

Scott Haynie

Kiran Shantaram Dalvi

Adonijah Park

Sahil Doshi

Suresh Babu Sajja

Shuvabrata Ganguly

Peter Ferrie

Mircea Ciubotariu

Shane Anil Pereira

Andrew James Jorgensen

Daniel Rabinovich

Peng Wu

Frank Harris

Stephen James Oglesby

Archana Padmasenan

Salman Ahmad Syed

Tal Eidelman

Sudheer Kumar Sanapati

Vaibhav Khunger

Edwin Ma

Shailesh Bhagwat

Ravi Mantrala

Rajkrishnan Parameswaran

Pavan Kumar Korlepara

Systems and methods are provided for tracking sensitive data within a plurality of storage units, where storage units include block storage volume(s) and/or 'snapshot(s)' representing values of every block of a block storage volume or device at a particular point in time. A computing device may be configured to receive a notification identifying a first tainted storage unit. Subsequently, a subset of the plurality of storage units may be identified. Additional tainted storage units can be identified as tainted within the subset. This identification may depend on whether the storage units contain data blocks in common with the first tainted storage unit. Remedial action may be taken for storage units identified as tainted.

Techniques for tracking and securing sensitive data in distributed storage networks

A file-level snapshot access service provides direct access to individual files included in a snapshot for virtual volume of a block-storage service without requiring a volume to be re-created from the snapshot, attached to a computing device, or mounted in a file system. For example, a user/client may directly retrieve individual files from specified snapshots via a user interface/API of the file-level snapshot access service. Additionally, the file-level snapshot access service is configured to provide a listing of files included in a given snapshot. In some embodiments, a file-level snapshot access service may provide direct access to individual files included in snapshots generated for other types of storage systems, such as an object-based storage system.

File-level snapshot access service

Systems and methods are described for use of reserved storage capacity at a cloud provider network extension, sometimes referred to as a provider substrate extension ('PSE') or “outpost”, as a data cache, thereby improving performance of the PSE. The reserved storage capacity may be reserved for a primary purpose, such as data replication in the event of failure of an in-use portion of the PSE's storage capacity. During non-failover operation, the reserved storage capacity may be otherwise unused. Embodiments of the present disclosure enable the reserved storage capacity, when not used for its primary purpose, to be used as a data cache to store information potentially used at the PSE.

Data caching in reserved storage capacity of cloud provider network extensions

Provided is a system for facilitating recovery of deleted computing resources in a cloud network environment. A centralized resource recovery service may communicate with a plurality of resource management services that are each configured to create, modify, or delete their respective computing resources such as storage volumes, databases, and compute instances. The resource recovery service may allow configuration of resource group policies such that deletion of grouped resources can be managed more effectively and efficiently. For example, in the event that a deleted resources matches multiple resource retention rules, the resource retention rule that encompasses multiple resource types may be used to place the deleted resource in a recoverable state so that resources of such multiple resource types can be managed according to the same resource retention.

Resource retention rules encompassing multiple resource types for resource recovery service

Provided is a system for facilitating recovery of deleted computing resources in a cloud network environment. A centralized resource recovery service may be in network communication with a plurality of resource management services that are each configured to create, modify, or delete their respective computing resources such as data storage volumes, databases, compute instances, and the like. The resource recovery service may be configured to receive a delete request associated with a resource managed by one of the resource management services, and cause the resource to be retained in a recovery bin based on the resource satisfying one of a plurality of resource recovery conditions used to manage resource recovery across the resource management services.

Resource recovery service

Systems and methods for efficient and secure management of encrypted 'snapshots' for a remote provider substrate extension ('PSE') of a cloud provider network substrate are provided. The PSE may request and obtain a snapshot from the cloud provider network substrate, restore a volume from the snapshot, make changes to data in the restored volume, and/or initiate the creation and storage of a new snapshot that includes incremental updates to the original snapshot to reflect the changes made to data in the volume. An encrypted snapshot stored within the cloud provider network substrate may be decrypted using a cloud provider key designed for internal use only, and then re-encrypted using a PSE-specific key before providing the snapshot to the PSE, thereby avoiding the sharing of the cloud provider internal use only key outside the cloud provider network substrate.

Snapshot management for cloud provider network extensions

Snapshot management across cloud provider network extension security boundaries

Multi-envelope encryption provides redundancy for highly-available storage of encrypted data. Data, such as a 'snapshot' representing values of every block of a block storage volume or device at a specific point in time, may be encrypted before storage to prevent unauthorized access to the data. To further protect the data and prevent unauthorized access to the data, additional security measures may be taken. Multiple copies of the data key that is to be used to decrypt the data may be encrypted and stored separately from the encrypted data as envelopes. The different envelopes may each be encrypted using envelope keys. If one envelope key is later lost or otherwise becomes unavailable, the encrypted data can still be accessed by using a different envelope key to recover the data key and decrypt the data.

Multi-envelope encryption system

Systems and methods are provided for efficient handling of user requests to access shared resources in a distributed system, which handling may include throttling access to resources on a per-resource basis. A distributed load-balancing system can be logically represented as a hierarchical token bucket cache, where a global cache contains token buckets corresponding to individual resources whose tokens can be dispensed to service hosts each maintaining a local cache with token buckets that limit the servicing of requests to access those resources. Local and global caches can be implemented with a variant of a lazy token bucket algorithm to enable limiting the amount of communication required to manage cache state. High granularity of resource management can thus enable increased throttle limits on user accounts without risking overutilization of individual resources.

Hierarchical token buckets

Systems and methods are provided for conducting incremental restore operations on block storage volumes using an object-based snapshot. A full restore from an object-based snapshot can include copying all blocks of a data set from the object-based snapshot to a destination volume. For high capacity volumes, full restores may take large amounts of time. Moreover, full restores may be inefficient where a destination volume already contains some data of the snapshot. Embodiments of the present disclosure provide for incremental restore operations, where a delta data set is transferred from the snapshot to the destination volume, representing data in the snapshot is not known to already exist on the volume or another available volume.

Incremental restore for volumes using object-based snapshots

Generally described, one or more aspects of the present application correspond to techniques for managing snapshots in a partially encrypted state and providing access to partially encrypted snapshots. These snapshot management techniques allow a snapshot that is being encrypted to be accessed prior to the encryption being completed. Such a snapshot may have some blocks that have been encrypted, and other blocks that have not yet been encrypted. In order to provide access to such a snapshot in a partially encrypted state, a system may allow the encryption status of the blocks in the snapshot to be checked at the block level instead of at the snapshot level (or at some other intermediary level therebetween), according to some embodiments. By doing so, the system can reduce the delays resulting from snapshots that are locked during the encryption process.

Partially encrypted snapshots

A system implements task contention reduction via policy-based selection. Tasks waiting to be performed are indexed in a task data structure that groups the tasks based on the resources to which the tasks pertain. Workers request batches of tasks for the workers to perform. A scan cycle includes building multiple batches of tasks by scanning the task data structure for a requesting worker. A policy (e.g., random or some other form of optimization) determines where the scan cycle starts in the data structure. Each batch of tasks is delivered to a worker along with a token that keeps the state of the scan cycle (e.g., where the scan cycle started, and where the next scan to build the next batch within the scan cycle begins). The worker returns the token with the next request for the next batch and the next batch is built based on the token's state information.

Task contention reduction via policy-based selection

Generally described, one or more aspects of the present application correspond to techniques for creating encrypted block store volumes of data from unencrypted object storage snapshots of the volumes. These encryption techniques use a special pool of servers for performing the encryption. These encryption servers are not accessible to users, and they perform encryption and pass encrypted volumes to other block store servers for user access. The encryption context for the volumes can be persisted on the encryption severs for as long as needed for encryption and not shared with the user-facing servers in order to prevent user access to encryption context.

Encryption by default in an elastic computing system

Growing community of inventors

Sammamish, WA, United States of America

Sandeep Kumar