Amazon Technologies, Inc.

Christopher Richard Jacques De Kadt

Allan Henry Vermeulen

Timothy Daniel Cole

John Michael Morkel

Tate Andrew Certain

Artem Danilov

Andrew Wayne Ross

Carlos Vara Callau

Aaron Gifford Freshwater

Benjamin Warren Mercier

Sayantan Chakravorty

James Alfred Gordon Greenfield

Jacob Adam Gabrielson

Andrew James Lusk

Michael Groenewald

Yevgeniy Mikhaylyuta

Jacobus Johannes Van Der Merwe

David R Richardson

Marc John Brooker

Matthew John Campagna

Matthew Shawn Wilson

Peter Nicholas Desantis

Marcin Piotr Kowalski

Douglas Stewart Laurence

Rebecca Claire Weiss

Yannis Papakonstantinou

John Merrill Phillips

Brad Eugene Marshall

Matthew James Eddey

Akhilesh Mritunjai

Anthony A Virtuoso

Timothy William Bray

Nikita Pinski

Dennis Tighe

Robin Alan Golden

Richard Shawn Bice

Marc Andrew Bowes

Neil Andrew Eriksson

Michael Benjamin Deardeuff

Jacobus Johannes Nicolaas Van Der Merwe

Chi Hai Ho

Atreya Srivathsan

Trevor Simon Roger Dyck

Phil Simko

Jerry Connolly

Philip Simko

A database management system receives a request to perform a transaction. The database management system commits the transaction, and in response to committing the transaction, generates a cryptographic hash based on an attribute of the transaction. The cryptographic hash is stored in a leaf-region of a hash tree. In response to a request to verify the transaction, signatures are retrieved from the tree based on a traversal of the tree to locate the node corresponding to the transaction. The retrieved signatures are used to verify the transaction.

Cryptographic verification of database transactions

Techniques for role-based permission delegation in a provider network. The techniques include an assuming service in the provider network sending a request to a temporary credential service in the provider network to assume a delegation role. The assuming service, acting in the delegation role, sending a request to the temporary credential service to assume the customer role in accordance with a down scoping policy. The assuming service, acting in the customer role, performing an action in a strict subset of actions on a customer resource. The techniques improve the operation of the provider network by allowing a permission to perform an action on the customer resource that is granted by the customer to a delegating service in the provider network to be delegated to the assuming service while complying with the access control principle of least privilege.

Role-based permission delegation in a provider network

A transaction request compliant with a first version of a journal schema of a multi-data-store storage system is received at a journal manager. The journal schema indicates attributes of data objects which may be materialized at various data stores of the system. The journal manager stores an entry in the system's journal if the transaction meets acceptance criteria. Writes indicated in the entry are materialized at the data stores after verifying that the entry is compliant with the journal schema. After verifying that member data stores have approved a proposed change to the journal schema, another entry indicating a different version of the journal schema is added to the journal. Client-side components of the system obtain the current version of the journal schema to prepare the transaction requests.

Schema management for journal-based storage systems

A journaled database system may comprise data nodes that maintain a collection of data structured in accordance with a schema. A change to the schema may be applied by a journal module while a journal consumer continues to operate using a prior version of the schema. A buffer may be formed and have stored therein records describing state change instructions according to the prior view of the schema. An index of correspondence between the records in the buffer and in the source journal may be maintained.

Ejournal transcoding

At a logging service, a representation of a transaction requested by a client is received. The transaction includes at least a first write operation directed to a data object at a first data store. Based at least in part on the contents of the representation and contents of a persistent log of the logging service, a determination is made that the requested transaction is to be committed. A log record corresponding to the first write operation is inserted into the persistent log. The first write is propagated to the first data store after the record has been inserted in the persistent log.

Multi-database log with multi-item transaction support

A materialization configuration request is received via a programmatic interface from a client of a journal-based multi-data-store database. The request indicates a partitioning rule to be used to select, for respective writes indicated in committed transaction entries of a journal, the materialization node at which the writes are to be stored. A control plane component of the database verifies that a set of materialization nodes corresponding to the partitioning rule has been established, and initiates the propagation of writes from the journal to the materialization nodes by respective write appliers.

Customer-related partitioning of journal-based storage systems

A storage service is configured to receive one or more instructions specifying transformations that are to be applied to data sets stored by the storage service when the data sets are made available outside of particular storage locations within the storage service. In response to triggering events that make the data sets available outside of the particular storage locations, the storage services causes the transformations to be performed on the data sets prior to the data sets being accessible at one or more destination locations outside of the particular storage locations where the data sets are stored. In some embodiments, the transformations are performed on hardware included in the storage service or are performed on external hardware at the direction of the storage service.

Storage service supporting data transformations

A query restriction descriptor indicating one or more attributes of a data object of a multi-data-store storage system is determined at a client-side component of the system. The query restrictions descriptor is used to prepare at least a read set descriptor of a proposed transaction. The read set of the proposed transaction includes a query with a predicate directed to the one or more attributes. A transformation function is applied to the predicate, and the result of the transformation is included in the read set descriptor. The proposed transaction is sent to a journal manager of the system, which performs read-write conflict detection using the read set descriptor and write set descriptors of one or more previously-committed transactions.

Consistency management using query restrictions in journal-based storage systems

A shared resource service allows multiple clients to agree on rules for accessing a shared resource (e.g., a shared database or a shared service). The shared resource service also allows the clients to make changes to the rules (e.g., via consensus or majority vote). The clients use the shared resource service to enforce the rules, without having to trust each other to follow the rules when accessing the shared resource. First, the clients agree on a set of initial rules for accessing the shared resource and a set of initial rules for making changes to the rules. After the rules are initialized, then the clients can begin using the shared resource in accordance with the agreed upon rules. In response to a request for accessing the shared resource or a request for changing rules, the shared resource service enforce the applicable rules.

Agreement and enforcement of rules for a shared resource

Methods and apparatus for profile-guided preloading for virtualized resources are described. A block-level storage volume whose contents are to be populated via data transfers from a repository service is programmatically attached to a compute instance. An indication of data transfers from the repository to a block storage service implementing the volume is obtained, corresponding to a particular phase of program execution at the compute instance. A storage profile is generated, based at least in part on the indication of data transfers. The storage profile is subsequently used to pre-load data from the repository service on behalf of other compute instances.

Profile-guided data preloading for virtualized resources

A provider network provides task capture and execution for tasks performed with respect to resources in the provider network. Task actions performed with respect to resources may be captured based on inputs to a provider network via a network-based service for the provider network in order to generate a recorded task. The recorded task may be stored for subsequent execution. In response to the detection of an execution event for the recorded task, the task actions may be performed as described in the recorded task. In some embodiments, task parameters and execution events may be defined for recorded tasks. Multiple recorded tasks may be linked together to be managed and executed as recorded task workflows. Recorded tasks may also be obtained that were recorded by other clients, or recorded for different provider networks.

Capture and execution of provider network tasks

A control-plane component of a network-accessible service receives an indication that a resource request has been submitted by a client. The component determines, based at least in part on metadata associated with the client, that a first parameter selection policy (PSP) of one or more PSPs created for the service is applicable. The first PSP includes a condition set to determine applicability, and a value set indicating parameter values to be used to fulfill resource requests to which the PSP applies. Using the value set, an effective parameter value list for one or more operations to be performed to allocate the requested resource is generated, and the allocation operations are initiated.

Policy-based allocation of provider network resources

A snapshot manager examines a set of entries of a journal of a multi-data-store storage system. Each entry is formatted in a data-store-independent transaction language and various entries contain a respective indication of a committed change to a data object or to a journal schema. Corresponding to various data objects that are materialized at least at a data store of the storage system, respective compact state change representations are generated based on the set of journal entries affecting the data objects. A given compact state change representation, also formatted in the transaction language, may represent cumulative effects of several changes. A snapshot of the journal, comprising the compact state representations, is stored.

Compact snapshots of journal-based storage systems

A data store manager of a multi-data-store storage system detects an error condition associated with applying a state change indicated in a particular journal entry of the storage system to a first data store. A first redaction entry inserted in the journal at some point after the particular entry is identified. The first redaction entry indicates the particular entry as a redaction target. After verifying that the journal does not include another redaction entry targeting the first redaction entry, processing of the particular entry is terminated and processing of later-inserted entries can proceed.

Storage system journal entry redaction

Growing community of inventors

Seattle, WA, United States of America

Christopher Richard Jacques De Kadt