Amazon Technologies, Inc.

Open Invention Network LLC

Citrix Systems, Inc.

Coho Data Inc.

Jacob Taylor Wires

Andrew Kent Warfield

Timothy John Deegan

Keir Fraser

Daniel Stodden

Brendan Anthony Cully

Christopher Clark

Dutch Meyer

Jean Maurice Guy Guyader

Geoffrey Lefebvre

Kevin Jamieson

Valeri Atamaniouk

Leon Thrane

Sidharth Bajaj

Nikolaos Koutsikos Tsatsalmas

Yuri Zarubin

Marc Stephen Olson

Christopher Magee Greenwood

Marcin Marek Wolicki

Julien Mascart

Seth William Markle

Wesley Shawn Davis

Ming Yiu Andy Siu

Mohammad Abdul-Amir

Jingmin Zhu

Jun Dai

Zhiyuan Lin

Systems and methods are provided for handling file operations from a hosted computing instance via a secure compute layer. The secure compute layer is presented to the instance as a virtualized service device that is locally addressable by the instance. Software within the instance can submit file operations to the virtualized service device, which the secure compute layer can translate into calls to a network-accessible storage service. Results from the calls can then be passed back to the instance through the virtualized service device. As a result, the instance can communicate with a variety of different network services, without itself implementing network communications for those services.

Offloading of remote service interactions to virtualized service devices

Systems and methods are provided for management of data access latency in a distributed storage system by dynamically replicating data objects experiencing a high request volume. When a storage node is unable to meet a data access latency target (e.g., due to one or more data objects hosted by the storage node being the subject of a large quantity of requests), the storage node may initiate a process by which a second storage node is employed to reduce the request processing load. The second storage node may cache a copy of the requested data object, and may service at least a portion of requests for the data object using the cached copy.

Data retrieval latency management system

Systems and methods are provided for implementing a multi-service file system for a hosted computing instance via a locally-addressable secure compute layer. Software within the instance can submit file operations to the secure compute layer, which the secure compute layer can translate into calls to one or more network-accessible storage services. To provide a multi-service file system, the secure compute layer can obtain mapping data mapping file system objects within the virtualized file system to different network-accessible storage services. On receiving a file operation, the secure compute layer can determine one or more network-accessible storage services corresponding to the file operation, and submit appropriate calls to the one or more network-accessible storage services. By varying the calls for file operations, various functionalities, such as data backup, write staging, read caching, and failover can be implemented independent of both operation of the hosted computing device and the network-accessible storage services.

Pseudo-local multi-service enabled file systems using a locally-addressable secure compute layer

Systems and methods are provided for exposing disparate storage systems and technologies to compute instances through a unified file system interface that is agnostic to the underlying storage systems and technologies. A file system namespace service may provide the ability to compose a file system from different storage technologies and systems (e.g., block storage, object storage, etc.). The file system namespace service may also provide the ability to layer different storage services (e.g., by treating local block storage volume as a write back cache, by treating a file system as a read-through cache in front of an object storage service, etc.), and/or the ability to bind virtual files to other compute instances and/or storage systems.

Composable network-storage-based file systems

Sets of asynchronous replication operations may be tracked to ensure consistency. A tracking service may receive notifications of pending asynchronous replication tasks, and responsive to receiving a manifest indicating a request to be notified upon completion of a set of pending replication asynchronous tasks, matches individual ones of the tasks within the set to tasks indicated as pending. The tracking service may then select a routing table based on a most recent sequence number associated with the set of tasks, determine one of more tracking nodes assigned to track the set of tasks, and send the manifest to each of the tracking nodes. As individual ones of the tasks complete, notifications of completion may be sent to the tracking nodes and an aggregator node aggregates the completion notifications for the set. Once all completion notifications are received, a response to the request indicating completion may be sent.

Granular tracking of replication consistency using subsets of asynchronous replication tasks

Dynamic management of data traffic workflows is performed. An event to perform a data traffic workflow at a remote performance location may be received. Computing resources to perform the data traffic workflow may be identified. Operations to perform the data traffic workflow may be dynamically directed by the identified computing resources to adaptively balance performance of the operations with operations for other data traffic workflows in order to meet respective performance requirements of the data traffic workflows.

Dynamically managed data traffic workflows

Systems, methods and devices for distributed memory management comprising a network component configured for network communication with one or more memory resources that store data and one or more consumer devices that use data, the network component comprising a switching device in operative communication with a mapping resource, wherein the mapping resource is configured to associate mappings between data addresses associated with memory requests from a consumer device relating to a data object and information relating to a storage location in the one or more memory resources associated with the data from the data object, wherein each data address has contained therein identification information for identifying the data from the data object associated with that data address; and the switching device is configured to route memory requests based on the mappings.

Systems, methods and devices for integrating end-host and network resources in distributed memory

A distributed computing system for automatic constraint-based data resource arrangement, comprising a plurality of computing components being communicatively coupled to each other, each computing component comprising the following data resources: data storage media for storing client-related digital information, a data processor for processing said client-related digital information, and a network communications interface for communicating said client-related digital information; and a constraint engine for automatically determining alternate arrangements of said data resource assignments, said constraint engine comprising a constraint processor and a constraint database, said constraint database for receiving and storing changeable digital constraint parameters indicative of permissible operational constraints on said data resources, wherein said alternate arrangements comply with at least a first set of said changeable digital constraint parameters; wherein said data resource assignments are reassigned from a current arrangement in accordance with a selected one of said alternate arrangements upon an operational change to said data storage system.

Methods, systems, and devices for adaptive data resource assignment and placement in distributed data storage systems

The performance of asynchronous tasks may be dynamically configured. An evaluation of pending tasks that includes updates for new tasks and completed tasks being removed is performed. A target capacity for the data traffic workflow is determined from the evaluation of pending tasks in order to satisfy a performance requirement for the data traffic workflow. Modifications to a performance configuration for the data traffic workflow are then made based on a comparison with the target capacity.

Dynamic performance configuration for data traffic workflows

Disclosed are methods and systems for of determining snapshot storage space consumption in a data storage system, a snapshot being a representation of a given state of a data object comprising of direct references to data blocks for data that has been changed from a prior state and indirect references to existing direct references in the prior state for unchanged data, the method comprising the steps: for each write of data to a given snapshot of the data object, storing unique address information relating to the unique direct references for said writes, said unique address information comprising at least a count of unique direct references; upon generating a further snapshot of a prior snapshot, retaining said prior snapshot and unique address information in storage as read-only access; and determining a count of unique direct references for all snapshots relating to the data object based on the stored unique address information.

Systems, devices and methods for determining and reclaiming snapshot storage consumption in data storage systems

A data transport service of a provider network provide isolated and flexible network data transmission between different computing infrastructure collections of the provider network (e.g., between different geographical regions) for different users. The data transport service may receive a request from a user for assignment of a channel. The request specifies a desired capacity of the channel to transmit data from a computing infrastructure collection to another a computing infrastructure collection. Based on the requested capacity for the channel, the data transport service may reserve, for the channel, network resource capacity to transmit the data between the infrastructures and reserve, for the channel, compute instance capacity to ingest the data and to send the data between the infrastructures using the reserved network resource capacity. The reserved network resource capacity and reserved compute instance capacity are dedicated to the channel and are unavailable for use by other channels.

Growing community of inventors

Vancouver, Canada

Jacob Taylor Wires