Amazon Technologies, Inc.

Tianming Zheng

Leo Parker Dirac

Sriram Krishnan

Charles Eric Dannaker

Sudhakar Rao Puvvadi

Daniel Blick

Tarun Agarwal

Jun Qian

Polly Po Lee

Ralf Herbrich

Jin Li

Michael Brueckner

Donghui Zhuo

Nicolle M Correa

Robert Matthias Steele

James Joseph Jesensky

Aleksandr Mikhaylovich Ingerman

Saman Zarandioon

Steven Andrew Loeppky

Rakesh Ramakrishnan

Polly Po Yee Lee

Consistency metadata, including a parameter for a pseudo-random number source, are determined for training-and-evaluation iterations of a machine learning model. Using the metadata, a first training set comprising records of at least a first chunk is identified from a plurality of chunks of a data set. The first training set is used to train a machine learning model during a first training-and-evaluation iteration. A first test set comprising records of at least a second chunk is identified using the metadata, and is used to evaluate the model during the first training-and-evaluation iteration.

Consistent filtering of machine learning data

A record extraction request for a data set is received at a machine learning service. A plan to perform one or more chunk-level operations (such as sampling, shuffling, splitting or partitioning for parallel computation) on chunks of the data set is generated. A set of data transfers that results in a particular chunk being stored in a particular server's memory is initiated to implement the first chunk-level operation of the sequence. A second operation such as another filtering operation or a feature processing operation is performed on a result set of the first chunk-level operation.

Input processing for machine learning

A determination that a machine learning data set is to be shuffled is made. Tokens corresponding to the individual observation records are generated based on respective identifiers of the records' storage objects and record key values. Respective representative values are derived from the tokens. The observation records are rearranged based on a result of sorting the representative values and provided to a shuffle result destination.

Consistent sort-based record-level shuffling of machine learning data

A request to split a data set comprising observation records located in a group of storage objects is received. With respect to a particular observation record, a token is generated based on an identifier of the record's storage object and a key value of the record. A numeric value is calculated using the token, and the observation record is assigned to a split subset using the numeric value. An indication of the assignment is provided to a destination associated with the split subset.

Consistent randomized record-level splitting of machine learning data

Growing community of inventors

Seattle, WA, United States of America

Tianming Zheng