Sun Microsystems, Inc.

Ab Initio Technology LLC

Oracle America, Inc.

Other

Ann M Wollrath

James H Waldo

Kenneth C R C Arnold

Robert W Scheifler

Peter C Jones

Roger S Riggs

Tim Wakeling

Ian Schechter

Brian T Murphy

Zane Pan

Kenneth C Arnold

Graham Hamilton

Marshall Alan Isman

Peter B Kessler

Bryan Phil Douros

Eduardo Pelegri-Llopart

Michael P Warres

Timothy Wakeling

Kenneth Crc Arnold

Bryan O'Sullivan

Adrian E Colley

Robert W Schiefler

In a first aspect, a method includes, at a node of a Hadoop cluster, the node storing a first portion of data in HDFS data storage, executing a first instance of a data processing engine capable of receiving data from a data source external to the Hadoop cluster, receiving a computer-executable program by the data processing engine, executing at least part of the program by the first instance of the data processing engine, receiving, by the data processing engine, a second portion of data from the external data source, storing the second portion of data other than in HDFS storage, and performing, by the data processing engine, a data processing operation identified by the program using at least the first portion of data and the second portion of data.

Processing data from multiple sources

An approach to parallel access of data from a distributed filesystem provides parallel access to one or more named units (e.g., files) in the filesystem by creating multiple parallel data streams such that all the data of the desired units is partitioned over the multiple streams. In some examples, the multiple streams form multiple inputs to a parallel implementation of a computation system, such as a graph-based computation system, dataflow-based system, and/or a (e.g., relational) database system.

Parallel access to data in a distributed file system

Use of a smart proxy as a wrapper around a stub in a distributed system. Instead of receiving a stub as a result of a remote procedure call, a caller receives a smart proxy including the stub as an embedded object. The smart proxy performs predefined processing associated with a remote procedure call, the processing possibly occurring before, during, or after a response to the call.

Downloadable smart proxies for performing processing associated with a remote procedure call in a distributed system

An improved lookup service is provided that allows for the dynamic addition and deletion of services. This lookup service allows for the addition and deletion of services automatically, without user intervention. As a result, clients of the lookup service may continue using the lookup service and its associated services while the updates occur. Additionally, the lookup service provides a notification mechanism that can be used by clients to receive a notification when the lookup service is updated. By receiving such a notification, clients can avoid attempting to access a service that is no longer available and can make use of new services as soon as they are added to the lookup service.

Dynamic lookup service in a distributed system

In accordance with the present invention a process is provided for allocating and deallocating resources in a distributed processing system having a requester platform and a server platform. The process involves receiving a request from the requestor platform referring to a system resource and specifying a requested lease period, permitting shared access to the system resource for a lease period, sending a return call to the requestor platform advising of the lease period, and deallocating the system resource when the lease period expires.

Methods, apparatus, and product for distributed garbage collection

A system consistent with the present invention reduces the number of redundant class descriptors that are sent during remote method calls by using serialization contexts. 'Serialization contexts' are dictionary objects that map a class descriptor to a corresponding integer handle. When possible, the integer handle, rather than the full class descriptor, is passed, saving processing time in RMI calls.

Method and system for passing objects in a distributed system using serialization contexts

A homogeneous execution environment operates within a heterogeneous client-server network. A client selects a server and transmits a procedure call with parameters. In response, a server dynamically and securely downloads code to a compute server; invokes a generic compute method; executes the code on the compute server; and returns the results to the calling client method, preserving the result on the compute server if requested. This technique is efficient in that it does not require multiple copies of code to be downloaded or compiled since server byte-codes can be executed on each of the different systems, therefore downloading or compiling multiple copies of code can be avoided. The code can be compiled once and downloaded as needed to the various servers as byte-codes and then executed.

Method and apparatus for dynamic distributed computing over a network

Use of a policy object for verification in a distributed system. A machine downloads a policy object containing a reference to code governing verification of data. The machine uses the reference to obtain the code and locally verify data or other information. As particular rules for the data change, the policy object may be updated to provide a reference to the code for the new rules when it is downloaded.

Apparatus and method for dynamically verifying information in a distributed system

A stub retrieval and loading subsystem is disclosed for use in connection with a remote method invocation system. The stub retrieval and loading subsystem controls the retrieval and loading of a stub for a remote method, into an execution environment, to facilitate invocation of the remote method by a program executing in the execution environment. The stub retrieval subsystem includes a stub retriever for initiating a retrieval of the stub and stub loader for, when the stub is received by the stub retriever, loading the stub into the execution environment, thereby to make the stub available for use in remote invocation of the remote method. In one embodiment, the stub retrieval and loading subsystem effects the retrieval and loading for a program operating in one address space provided by one computer, of stub class instances to effect the remote invocation of methods which are provided by objects operating in another address space, which may be provided by the same computer or a different computer.

System and method for facilitating dynamic loading of stub information to enable a program operating in one address space to invoke processing of a remote method or procedure in another space

A distributed computer system uses a single interface at the client site to handle calls to call both active and passive remote objects. Accordingly, the calling process does not need to be aware of distinctions between active and passive objects. Further, remote objects are aggregated into common groups of objects, thereby providing greater security between objects of disparate groups and efficiency between related objects of the same group. Preferably, different groups are run on different Java virtual machines.

Growing community of inventors

Groton, MA, United States of America

Ann M Wollrath