Amazon Technologies, Inc.

Yohan Mammen

Roshan Ram Rammohan

Jonah Beckford

Shantanu Shailendrakumar Fauji

Sivaranjini Dharmalingam

Sean Gwizdak

Nathan Thomas Close

Jiahui Jiang

Technologies are disclosed for mapping documents to candidate duplicate documents in a document corpus. A bitset optimized inverted index is created for a document corpus. A document is received for which candidate duplicate documents in the document corpus are to be identified. The document is tokenized using adaptive tokenization. A determination made as to whether tokens in the document are represented in the bitset optimized inverted index. A list of candidate duplicate documents is created for tokens represented in the optimized inverted index utilizing in-memory bitsets that map tokens to documents that contain the tokens in the document corpus.

Optimized mapping of documents to candidate duplicate documents in a document corpus

Disclosed are various embodiments for the management of dependencies on data. Multiple jobs are executed in one or more computing devices. It is determined that a first one of the jobs consumes data produced by a second one of the jobs, and a dependency is recorded. It is then determined whether the dependency is satisfied. When the dependency is determined to be unsatisfied, execution of the first one of the jobs is suspended. When the second one of the jobs produces the data, the dependency is updated as being satisfied. Execution of the first one of the jobs is resumed in response to the dependency being updated as being satisfied.

Managing data dependencies among multiple jobs using separate tables that store job results and dependency satisfaction

Growing community of inventors

Renton, WA, United States of America

Yohan Mammen