Parallel Machines Ltd.

Datarobot, Inc.

Lior Amar

Lior Khermosh

Gal Zuckerman

Avner Braverman

Michael Adda

Eli Finer

Nisha D Talagala

Swaminathan Sundararaman

Sriram Subramanian

Dan Aloni

Bharath Ramsundar

Sindhu Ghanta

Drew Roselli

Vinay Sridhar

Apparatuses, systems, program products, and method are disclosed for machine learning abstraction. An apparatus includes an objective module configured to receive an objective to be analyzed using machine learning. An apparatus includes a grouping module configured to select a logical grouping of one or more machine learning pipelines to analyze a received objective. An apparatus includes an adjustment module configured to dynamically adjust one or more machine learning settings for a logical grouping of one or more machine learning pipelines based on feedback generated in response to analyzing a received objective.

Machine learning abstraction

Various embodiments of systems and methods to efficiently use a compute element to process a plurality of values distributed over a plurality of servers using a plurality of keys. In various embodiments, a system is configured to identify (or 'derive') the various server locations of various data values, to send requests to the various servers for the needed data values, to receive the data values from the various servers, and to process the various data values received. In various embodiments, requests are sent and data values are received via a switching network. In various embodiments, the servers are organized in a key value store, which may optionally be a shared memory pool. Various embodiments are systems and methods with a small number of compute elements and servers, but in alternative embodiments the elements may be expanded to hundreds or thousands of compute elements and servers.

Systems and methods for cache streams

Described herein are systems and methods to process efficiently, according to a certain order, a plurality of data sets arranged in data blocks. In one embodiment, a first compute element receives from another compute element a first set of instructions that determine an order in which a plurality of data sets are to be processed as part of a processing task. Relevant data sets are then streamed into a cache memory associated with the first compute element, but the order of streaming is not by order of storage but rather by the order conveyed in the first set of instructions.

Systems and methods for processing data sets according to an instructed order

Various systems and methods to perform efficiently a first processing task in conjunction with a plurality of data sets. A first code sequence comprises a plurality of general commands, and a specific command including a description of a first data processing task to be performed in conjunction with the data sets. The general commands are received and processed in a standard manner. The specific command is identified automatically by its nature, and the description within the specific command is then converted into a first sequence of executable instructions executable by a plurality of compute elements holding the plurality of data sets. The ultimate result is an efficient implementation of the first processing task. In some embodiments, the implementation of the first processing task is assisted by a pre-defined procedure that allocates the data sets to the compute elements and shares instances of executable instructions with the compute elements.

Systems and methods for efficiently implementing functional commands in a data processing system

Described herein are systems and methods to prevent a controller in a DDIO (data direct input output) system from shifting currently-required data out of a cache memory. In one embodiment, a compute element disables caching of some specific addresses in a non-cache memory, but still enables caching of other addresses in the non-cache memory, thereby practically disabling the DDIO system, so that data sets not currently needed are placed in the addresses in the non-cache memory which are not cached. As a result, currently-required data are not shifted out of cache memory. The compute element then determines that the data sets, which formerly avoided being cached, are now required. The system therefore copies the data sets that are now required from addresses in non-cache memory not accessible to cache memory, to addresses in non-cache memory accessible to cache memory, thereby allowing the caching and processing of such data sets.

Utilizing a cache mechanism by copying a data set from a cache-disabled memory location to a cache-enabled memory location

Various systems and methods to generate automatically a procedure operative to distributively process a plurality of data sets stored on a plurality of memory modules. Under the instruction of the automatically generated procedure, compute elements request data sets relevant to a particular task, such data sets are fetched from memory modules by data interfaces which provide such data sets to the requesting compute elements, and the compute elements then process the received data sets until the task is completed. Relevant data sets are fetched and processed asynchronously, which means that the relevant data sets need not be fetched and processed in any particular order.

Systems and methods for automatic generation of parallel data processing code

Described herein are systems and methods to execute efficiently a plurality of actions, in which multiple actions require the use of a single data set. The data set is fetched from a data source, across a switching network, to a memory associated with a first compute element. This is the only fetching of the data set from the data source, and the only fetching across a switching network, thereby minimizing fetching across the switching network, reducing the load on the switching network, decreasing the time by which the data set will be accessed in second and subsequent processes, and enhancing the efficiency of the system. In some embodiments, processes are migrated from second and subsequent compute elements to the compute element in which the data set is stored. In some embodiments, second and subsequent compute elements access the data set stored in the memory associated with the first compute element.

Systems and methods for executing actions using cached data

Various systems and methods to generate automatically a procedure operative to divide a processing task between two or more compute elements. A first compute element converts a code sequence into a sequence of executable instructions, which direct a second compute element to perform a first processing sub-task on a data set, and which also direct a third compute element to perform a second processing sub-task on the data set modified by the first processing sub-task. A memory module storing the data set may be embedded in a server with at least one of the compute elements. In some of the embodiments, all of the compute elements are part of a single system, whereas in alternative embodiments, at least some of the compute elements are part of two or more sub-systems.

Systems and methods for automatic generation of task-splitting code

Described herein are various systems and methods to automatically decide to aggregate data write requests in a distributed data store. A system initiates outgoing data write requests in synchronization with incoming data store commands, thereby facilitating low-latency read-back of the data. In response to an absence of data read requests, the system automatically changes such that each request includes two or more data sets, thereby breaking synchronization but consequently reducing traffic load on a switching network within the system. If the system later detects data read requests for previously stored data, the system will automatically change back to the original synchronized state, thereby decreasing the latency of accessing stored data. The system alternates between the modes of operation to achieve balance between low latency of data access and reduced traffic load on the switching network.

Growing community of inventors

Nes Tsiona, Israel

Lior Amar