The Mathworks, Inc.

Yashwanth Annapureddy

Jocelyn Martin

Penelope Anderson

Thomas Lane

James Stewart

Richard Amos

Wei Wang

Michael Procopio

Aaditya Kalsi

Anandan Rangasamy

Nicholas Haddad

Kari Sortland

According to some embodiments, a method for making input data available for processing by one or more processors comprises storing one or more parameters, wherein the one or more parameters comprise information identifying a location of the input data; and creating a datastore object using the one or more parameters, wherein the datastore object interfaces the input data and includes a read method for reading a chunk, the chunk being a subset of the input data, and having a size that does not exceed a memory size assigned to the one or more processors. According to some embodiments, the one or more parameters further comprise one or more of a type of the input data; a format of the input data; an offset for reading from the input data; a size of the chunk; a condition for determining the chunk; and a query for deriving the input data.

Datastore mechanism for managing out-of-memory data

According to some embodiments, a method for processing input data comprises creating a MapReducer object corresponding to a MapReduce environment; and receiving, by a MapReduce interface, a plurality of input parameters comprising the input data; a mapper function; a reducer function; and the MapReducer object; and using the MapReduce interface to process the input data by one or more processors in the MapReduce environment using the mapper function and the reducer function. According to some embodiments, the method further comprises creating a second MapReducer object, wherein the second MapReducer object corresponds to a second MapReduce environment; receiving, by the MapReduce interface, the second MapReducer object in place of the first MapReducer object; and utilizing the MapReduce interface to process the input data by the one or more processors in the second MapReduce environment using the mapper function and the reducer function.

Unified mapreduce framework for large-scale data processing

Growing community of inventors

Waltham, MA, United States of America

Yashwanth Annapureddy