Emc IP Holding Company LLC

Amazon Technologies, Inc.

Srikanth Satyanarayana

Andrei Paduroiu

Alan Stuart Robbins

Richard Michael Sears

Thomas Kaitchuck

Aspects of the present disclosure relate to managing data storage resources. In embodiments, one or more data streams are received. Each data stream can include one or more data portions. Further, one or more storage parameters are monitored. Each storage parameter can include data input load of each data stream, data write rate of each stream, number of events in each stream, and data ingestion rates of one or more storage devices. Data storage resources are elastically scales based on any changes to the at least one monitored storage parameter.

Data storage management and resource scaling

Implementations are provided herein for atomically committing related stream data across multiple, distributed resources. Transactions can be established that are distributed across multiple hosts, and their data can be made to appear atomic to an observing process. A master status for the transaction can be used to flag to other processes that the transaction is being committed. A stream to which the transaction is being appended to can be locked until the transaction data is committed in full. It can be appreciated that one an event to a transaction is acknowledged, and the transaction is merged, its commitment is guaranteed.

Atomically committing related streaming data across multiple distributed resources

Disclosed are various embodiments for scalable workflow processing. In one embodiment, a plurality of decision applications each embody a functional expression of a corresponding one of a plurality of workflow definitions. Each of the plurality of decision applications is configured to be repeatedly instantiated. A workflow execution application is configured to orchestrate an execution of each one of a plurality of workflow instances. The workflow execution application is in data communication with each instance of a respective one of the plurality of decision applications. Each of the plurality of decision applications is configured to determine a next action to be taken for respective ones of the plurality of workflow instances.

Scalable workflow processing

Implementations are provided herein for managing streaming data that appended and multiplexed into a durable write ahead replicated log. By writing data in the durable log, large amounts of small writes can be processed quickly. Data in the durable log can be de-multiplexed and packaged into segment containers. Segment containers can serialize stream segment specific data and can be stored in long term storage. Data that has been stored in long term storage can be truncated from the durable write ahead log making room for new data.

De-multiplexing streaming data from a fast write ahead durable log into segments

Disclosed are various embodiments for scalable workflow processing. To this end, a workflow instance is placed in a queue accessible to a processor-based system, the queue being associated with a workflow. A workflow history associated with the workflow instance is applied to a decision application in response to a poll from the decision application, where the decision application is configured to determine a next action to be taken for the workflow instance. The next action to be taken for the workflow instance is initiated in the processor-based system based on a directive from the decision application generated based on the workflow history.

Growing community of inventors

Issaquah, WA, United States of America

Srikanth Satyanarayana