Amazon Technologies, Inc.

Hewlett-packard Development Company, L.p.

Hewlett Packard Enterprise Development LP

Microsoft Technology Licensing, LLC

Other

Amiato, Inc.

Mehul A Shah

Anurag Windlass Gupta

Benjamin Albert Sowell

Prajakta Datta Damle

George Steven McPherson

Gopinath Duddi

Stavros Harizopoulos

Xiaozhou Li

Bohou Li

Nathan A Binkert

Robert S Schreiber

Douglas Brian Terry

Nathan Lorenzo Binkert

Eric A Anderson

Anthony A Virtuoso

Wojciech Golab

Supratik Chakraborty

Arif Abdulhusein Merchant

Marcos Kawazoe Aguilera

Vijayan Prabhakaran

John Johnson Wylie

Indrajit Roy

Kevin R Meyer

Dimitris Tsirogiannis

Bryan D Kaplan

Dimitrios Tsirogiannis

Parthasarathy Ranganathan

Andrew Edward Caldwell

Christos Karamanolis

Goetz Graefe

Jeffrey Clifford Mogul

Robert E Tarjan

Ram Swaminathan

Janet Lynn Wiener

Rahul Sharma Pathak

Mehul Y Shah

Stephen Michael Ash

Timothy Jones

Santosh Chandrachood

Andrew Borthwick

Juliana Saussy

Abhishek Sharma

Bijay Singh Bisht

Pedrito Uriah Maynard-Zhang

Shehzad Qureshi

Sergei Dobroshinsky

Chethan Kommaranahalli Rudramuni

Adam Lawrence Joseph Heinermann

Alaykumar Navinchandra Desai

David Andrew Roberts

Alan Hersh Karp

Paolo Faraboschi

Naveen Muralimanohar

Jakub Kulesza

Jichuan Chang

William G Horne

Alistair Veitch

Jason Douglas Denton

Jian Fang

Himanshu Lal

Carlos Garcia-Alvarado

Alistar Veitch

Linchi Zhang

Marco DiPlacido

Bin Lu

Eric Mills

Wesley Blumenthal

Krishnaditya Kandregula

Sachet Saurabh

Vinay Singh

Zhuzeng Lu

James Thomas Kiraly

Shivaram Venkataraman

Amitanand Aiyer

Eduardo Argollo De Oliveira Dias, Jr

Alina Ene

Nikola Milosavljevic

Sandhya Edupuganti

Dima Belikov

Akila Tennakoon

Seth Thomas Denney

Vinay Agrawal

Vikas Malik

Tushar Poddar

Mukesh Punhani

Shashank Bhardwaj

Roman Gavrilov

Brian Scott Ross

Linan Zheng

Stavros Hartzopoulos

Matt Bachman

Mohit Saxena

Michael Sklar Chess

Grayson Osburn Vigeant Clendenon

Frank Benkstein

Abhijit Uday Pujare

John Sontag

Wojclech Golab

Daniel L Reed

Rahul Pathak

Respective delegation records indicating that a first access controller and a second access controller have been authorized to grant access to a respective set of cells of a table of a storage management service are stored. In response to receiving an access request of a data accessor, permission records indicating that the access controllers have granted permissions to the data accessor to respective subsets of the table's cells are identified. A collection of cells of the table to which the data accessor has been granted access permission is identified using the permission records, and a response to the access request is generated using the collection.

Delegated fine-grained access control for data lakes

Code may be dynamically routed to computing resources for execution. Code may be received for execution on behalf of a client. Execution criteria for the code may be determined and computing resources that satisfy the execution criteria may be identified. The identified computing resources may then be procured for executing the code and then the code may be routed to the procured computing resources for execution. Permissions or authorization to execute the code may be shared to ensure that computing resources executing the code have the same permissions or authorization when executing the code.

Dynamically routing code for executing

Techniques for self-tuning an analytics system via observed execution optimization are described. Upon a need for execution resources, a resource manager can select a type of executor from multiple candidate executor types based at least in part on one or more of current execution data associated with the execution of tasks of a user application and/or historic execution data associated with one or more other applications. The current execution data may include event log data originated by the driver application based on the execution of the user application and/or metric data describing characteristics of one or more worker nodes involved with executing the user application or characteristics of one or more other executors implemented by the one or more worker nodes in executing the user application.

Self-tuning analytics system with observed execution optimization

Techniques for a distributed data processing application service in a cloud provider network are described. A virtual machine bundle is obtained, the virtual machine bundle including a copy of a memory and one or more registers of a paused virtual machine executing a distributed data processing application runtime. A request to launch a virtual machine to execute a distributed data processing application runtime is received, the request to launch the virtual machine including a bundle identifier that identifies the virtual machine bundle. The virtual machine is launched based at least in part on the virtual machine bundle. The virtual machine sends a message to register the virtual machine with a cluster of virtual machines.

Distributed data processing application service in a provider network

Extract, Transform, Load (ETL) processing may be initiated by detected events. A trigger event may be associated with an ETL process apply one or more transformations to a source data object. The trigger event may be detected for the ETL process and evaluated with respect to one or more execution conditions for the ETL process. If the execution conditions for the ETL process are satisfied, then the ETL process may be executed. At least some of the source data object may be obtained, the one or more transformations of the ETL process may be applied, and one or more transformed data objects may be stored.

Event driven extract, transform, load (ETL) processing

Specified performance attributes may be used to configure machine learning transformations for ETL jobs. Performance attributes for a machine learning pipeline that applies a model to as part of a transformation for an ETL job may be used to configure a parameter in a stage of the machine learning pipeline. The configured stage may then be used when training the model. The trained machine learning pipeline may then be applied as part of a transformation operation included in an ETL job performed by the ETL system.

Using specified performance attributes to configure machine learning pipepline stages for an ETL job

Recognizing unknown data objects may be implemented for data objects stored in a data store. Data objects that are identified as unknown may be accessed to retrieve a portion of the data object. Different representations of the data object may be generated for recognizing different data schemas. An analysis of the representations may be performed to identify a data schema for the unknown data object. The data schema may be stored in a metadata store for the unknown data object.

Recognizing unknown data objects

A protocol for implementing ACID transactions that provides snapshot isolation in a distributed setting that does not require synchronized clocks is described. The protocol ensures at commit time that transactions touching common objects do not commit out of order. The protocol can be used in the context of a distributed data lake built on an object store in which clients can transactionally add or remove objects from logical tables.

Transactional protocol for snapshot isolation without synchronized clocks

Data transformation workflows may be generated to transform data objects. A source data schema for a data object and a target data format or target data schema for a data object may be identified. A comparison of the source data schema and the target data format or schema may be made to determine what transformations can be performed to transform the data object into the target data format or schema. Code to execute the transformation operations may then be generated. The code may be stored for subsequent modification or execution.

Generating data transformation workflows

Techniques for storage optimization in a distributed object store are described. A storage optimization service of a provider network monitors changes to data objects in a distributed object store that are part of a data lake and are referenced by a table index. The storage optimization service determines whether particular storage optimizations involving the data objects would be beneficial, prioritizes the ordering of these optimizations with a focus on performing impactful optimizations first, while intelligently scheduling the optimizations to avoid overutilization of available resources.

Storage optimization in a distributed object store

Techniques for using tree data structures to maintain a transactionally consistent set with support for time-travel queries are described. When a transaction commits, a new version of the tree data structure is created using a copy-on-write based method such that the tree shares internal nodes with previous trees to save space. This approach may be used in the implementation of a transactional data catalog in which the files that make up a table are stored in a transactional set.

Tree-based approach for transactionally consistent version sets

Dynamic generation of data catalogs may be implemented for accessing data sets in different storage locations. Data sets may be accessed in order to extract portions of data. Structure recognition techniques may be applied to the extracted data in order to determine structural information for the data sets. The structural information may then be stored as part of a data catalog for the data sets. Requests to access the data catalog from different clients may be received and the requested structural data supplied so that the clients may access different data sets utilizing the supplied structural data. Data catalogs may be updated as changes to data sets are made.

Dynamic generation of data catalogs for accessing data

Techniques for implementing systems using transactional version sets are described. Transactional version sets or t-sets include a collection of elements, each having a collection of metadata. A t-set is transactional in that a sequence of updates to one or more t-sets are made within an atomic transaction. A t-set is versioned since each committed transaction that updates it produces a new timestamped version that can be accessed via time-travel queries.

Transactional version sets

Growing community of inventors

Saratoga, CA, United States of America

Mehul A Shah