Amazon Technologies, Inc.

Microsoft Technology Licensing, LLC

Laurion Darrell Burchall

Anurag Windlass Gupta

Pradeep Jnana Madhavarapu

Neal Thomas Fachan

Christopher Richard Newcombe

James McClellan Corey

Samuel James McKelvie

Benjamin Tobler

Colm MacCarthaigh

Grant Alexander Macdonald McAlister

Alexander Wetmore

Brett A Shirley

Jorgen Lawrence Johnson

Nicholas Ryman Vogel

Andrew E Goodsell

Ian Jose

Matthew David Allen

Matthew G Gossage

Jonathan Liem

James Mcclellan Corey

Samuel James Mckelvie

Neal Fachan

A database system may include a database service and a separate distributed storage service. The database service (or a database engine head node thereof) may be responsible for query parsing, optimization, and execution, transactionality, and consistency, while the storage service may be responsible for generating data pages from redo log records and for durability of those data pages. For example, in response to a write request directed to a particular data page, the database engine head node may generate a redo log record and send it, but not the data page, to a storage service node. The storage service node may store the redo log record and return a write acknowledgement to the database service prior to applying the redo log record. The server node may apply the redo log record and other redo log records to a previously stored version of the data page to create a current version.

Database system with database engine and separate distributed storage service

A distributed database system may implement fast crash recovery. Upon recovery from a database head node failure, a connection with one or more storage nodes of a distributed storage system storing data for a database implemented by the database head node may be established. Upon establishment of the connection with the storage nodes, that database may be made available for access, such as for various access requests. In various embodiments, redo log records may not be replayed in order to provide access to the database. In at least some embodiments, the storage nodes may provide a current state of data stored for the database in response to requests.

Fast crash recovery for distributed database systems

Nodes of a database service may receive a read request to perform a read of a record stored by the database service and a transaction request to perform a transaction to the record. First and second indications of time may be associated with the read and transaction, respectively. A potential read anomaly (e.g., fuzzy read, read skew, etc.) may be detected based, at least in part, on a determination that the first indication of time is within a threshold value of the second indication of time. In response to detecting the potential read anomaly, the read may be performed after the transaction specified by the transaction request, regardless of whether the first indication of time is indicative of an earlier point in time than the second indication of time.

Transaction ordering

A database system may maintain a plurality of log records at a distributed storage system. Each of the plurality of log records may be associated with a respective change to a data page. Upon detection of a coalesce event for a particular data page, log records linked to the particular data page may be applied to generate the particular data page in its current state. Detecting the coalesce event may be a determination that the number of log records linked to the particular data page exceeds a threshold.

System-wide checkpoint avoidance for distributed database systems

Methods and apparatus for a primary role reporting service for resource groups are disclosed. A reporting intermediary assigned to a resource group of a network-accessible service receives role information indicating the identity of a primary member of the group from a state manager of the network-accessible service. The intermediary receives a health status query pertaining to the resource group from a health checking subsystem used by a network address discovery service, and provides a health status response based on the role information. The response provided by the reporting intermediary is used by the network address discovery service to provide a network address in response to a lookup query for the resource group.

Primary role reporting service for resource groups

A database system may implement database cache survivability across database failures. In various embodiments, a database cache may be maintained independent of a failure of a database. A database cache may be maintained in a non-volatile memory device or maintained in a shared memory segment of system memory. Upon recovery from a database failure, a recovery point may be determined that indicates a consistent state of the database. Cache entries of the database cache inconsistent with the consistent state of the database may be invalidated, and the database cache may be made available for access requests directed toward the database. Valid cache entries from before the database failure may be made available without accessing a back-end data store for the database.

Database cache survivability across database failures

A continuous set of committed transactions can be lost without destroying the integrity of the database, by deferring the writing of the database pages stored in cache to the database on stable storage. A waypoint tracks a point in the transaction log following which transactions have not been applied to the database on stable storage and thus records after the waypoint are not needed for recovery to a consistent database. When there is a failover of an active database on Node A to a passive database on Node B, log records are compared starting with the most current and moving backwards and the point at which divergence in the log occurs is determined. When the lost portion of the log occurs after the point indicated by after the waypoint, the database is recoverable by log-based incremental reseed.

Delaying database writes for database consistency

Architecture for replicating a database through log shipping. A replication service replicates data using one or more new internal application program interfaces (APIs), a replay configuration API which abstracts configuration information for each destination (or target) instance. APIs for log file management as part of the replication process are provided for opening communications for log shipping, tracking success of log shipping and closing out log shipping communications. Log shipping APIs are also provided for log truncation control. A ship control object is provided as a base class which includes an API for other components of the replication service to operate on log files when the log files are placed into a log directory. The ship control API is utilized by the replication service and for interaction with storage system log files.

Interfaces for high availability systems and log shipping

A computing system stores a database comprising pages. Each of the pages is the same size. When a page is requested, a block of virtual memory addresses is associated with the page and a set of physical data storage locations is committed to the block of virtual memory addresses. A copy of the page is then stored into the set of physical data storage locations. Physical data storage locations committed to the virtual memory addresses associated with available free space in the copy of the page are deallocated, thereby allowing reuse of these physical data storage locations. A reference to the copy of the page is then returned.

Growing community of inventors

Seattle, WA, United States of America

Laurion Darrell Burchall