Amazon Technologies, Inc.

Yangbae Park

Daniel Opincariu

Jason Scott Flittner

Aaron John Seldon Steers

Fletcher Liverance

Siddharth Sridhar

Zhuonan Song

Sanjay Mathew Thomas

Stuart William Barnes

Laxmi Siva Prasad Balaramaraju Jalumari

Systems, devices, and methods are provided for implementing shadow data lakes. In at least one embodiment, a deletion workflow obtains a deletion request from a delete request cache service, gets attestation details from an attestation service, submits a job to scan one or more records from a source table of a data lake and publish the one or more records to a deleted records table of a shadow data lake, and cause deletion of the one or more records from the data lake.

Shadow data lakes

Methods, systems, and computer-readable media for indexing partitions using distributed Bloom filters are disclosed. A data indexing system generates a plurality of indices for a plurality of partitions in a distributed object store. The indices comprise a plurality of Bloom filters. An individual one of the Bloom filters corresponds to one or more fields of an individual one of the partitions. Using the Bloom filters, the data indexing system determines a first portion of the partitions that possibly comprise a value and a second portion of the partitions that do not comprise the value. Based (at least in part) on a scan of the first portion of the partitions and not the second portion of the partitions, the data indexing system determines one or more partitions of the first portion of the partitions that comprise the value.

Indexing partitions using distributed bloom filters

A data analysis platform may be based on database views. A build module may receive, from a source code repository, information about a modified definition of a view. The build module may identify schema objects on which the view depends and form instructions for creating the view and the schema objects. The instructions may be executed to form the updated version of the view in a schema space separate from a production schema space. A deployment pipeline may coordinate replacing the production version of the view with the new version in response to validating the new version of the view.

View-based data mart management system

A data validation service may validate data sets maintained for one or more data sources. Several rule sets may describe various rules used to validate one or more data sets. The rule sets may be automatically applied to respective data sets in order to validate the respective data sets according to a dynamically determined schedule for the application of the rule sets. Reporting events may be detected which correspond to a rule set. In response to detecting a reporting event, a responsive action may be performed as described in the rule set, such as providing notification of the reporting event.

Growing community of inventors

Seattle, WA, United States of America

Yangbae Park