Amazon Technologies, Inc.

Chance Ackley

Fletcher Liverance

Dominic Corona

Long Xuan Nguyen

Kevin Yavno

Methods, systems, and computer-readable media for a transformation specification format for multiple execution engines are disclosed. A transformation specification is expressed according to a transformation specification format. The transformation specification represents a polytree or graph linking one or more data producer nodes, one or more data transformation nodes, and one or more data consumer nodes. An execution engine is selected from among a plurality of available execution engines for execution of the transformation specification. The execution engine is used to acquire data from one or more data producers corresponding to the one or more data producer nodes, perform one or more transformations of the data corresponding to the one or more data transformation nodes, and output one or more results of the one or more transformations to one or more data consumers corresponding to the one or more data consumer nodes.

Transformation specification format for multiple execution engines

Techniques are generally described for adding transactional support to a distributed storage environment. In various examples, first data may be written to a first set of locations in a distributed computer-readable non-transitory storage system through a non-transactional file system interface. In various further examples, metadata associated with the first data may be generated during the writing of the first data. In some examples, the metadata may be stored associated with the first data in at least a second location in a second computer-readable non-transitory memory. In some examples, a manifest may be generated defining a transactional commit of at least a portion of the first data. In some examples, the manifest may be generated by processing the metadata using first committer logic. In some further examples, the manifest may be stored in a third computer-readable non-transitory memory.

Non-native transactional support for distributed computing environments

Methods, systems, and computer-readable media for lottery-based resource allocation with capacity guarantees are disclosed. A job request is received from a first client. The job request is submitted to a capacity management system that schedules jobs in a pool of compute resources. The pool comprises a first quantity of one or more slots and a second quantity of one or more slots. The second quantity is associated with a capacity guarantee for a second client. It is determined that the first quantity of one or more slots are in use by one or more jobs initiated prior to receiving the job request. It is determined that the second quantity of one or more slots comprises an available slot. The available slot is allocated to the job request.

Growing community of inventors

Royal Oak, MI, United States of America

Chance Ackley