Amazon Technologies, Inc.

George Shipilov

Kerry Quintin Lee

Matthew James Eddey

A system and method for maintaining data integrity during data transformation operations. The system and method include obtaining a message from a set of queues, obtaining, from a first data store, a data object indicated by the message, and generating, at a first host, a set of error detection codes corresponding to a transformation of the data object according to a transformation scheme. The system and method further include, transforming, at a second host different from the first host, the data object according to the transformation scheme into the transformation of the data object, verifying the transformation against the set of error detection codes, and storing the transformation in a second data store.

Ensuring data integrity during large-scale data migration

A system and method for using a dead letter queue to regulate host computing systems performing transformation or migration operations. The system and method including reading a message from a queue, the message including one or more references to a set of data objects, with the set of data objects including one or more data objects selected for a transformation operation according to a transformation scheme. The system and method further including performing a mitigation action upon determination that the message is one of a number of unsuccessfully processed messages in the queue that are associated with the computer system, when the number of unsuccessfully processed messages exceed a threshold.

Growing community of inventors

Seattle, WA, United States of America

George Shipilov