Cornell University

Reliable Network Solutions

Other

Microsoft Technology Licensing, LLC

Exostellar, Inc.

Robbert Van Renesse

Hakim Weatherspoon

Zhiming Shen

Johannes Gehrke

Maofan Yin

Fred Schneider

Mark Hayden

Emin Gün Sirer

Kevin Sekniqi

A method for instantiating and transparently migrating executing containerized processes includes receiving, by a container engine executing on a first machine, an instruction to instantiate a container image on the first machine, the container image including at least one process without including an operating system kernel. The container engine transmits, to a modified container runtime process, the instruction to instantiate the container image on the first machine. The modified container runtime process generates, on the first machine, a shim process representing the instantiated container image. The shim process forwards the instruction to an agent on a second machine. The agent directs instantiation of the container image, which includes providing, to the containerized process, at least one instruction for communicating with an operating system kernel of the second machine. A scheduler component on the first machine determines to migrate the containerized process to a third machine and directs the migration.

Methods and systems for instantiating and transparently migrating executing containerized processes

A method for dynamically optimizing and modifying allocation of virtual resources to processes includes allocating, by a first hypervisor, a first amount of a first virtual resource to a process executing in a virtual machine on a first machine. An agent, in communication with a resource allocation process executing in the virtual machine, determines a second amount of the first virtual resource to be utilized by the process. A scheduler receives an identification of the determined second amount and directs migration of the process to a second machine. A second hypervisor executing on the second machine allocates a third amount of the first virtual resource, the third amount substantially similar to the second amount. The method includes communicating, by the resource allocation process, to the process, that the first amount of the first virtual resource is allocated to the at least one process.

Methods and systems for dynamically optimizing and modifying allocation of virtual resources to processes

A method for instantiating and transparently migrating executing containerized processes includes receiving, by a container engine executing on a first machine, an instruction to instantiate a container image on the first machine. The container engine transmits, to a modified container runtime process, executing on the first machine, the instruction to instantiate the container image on the first machine. The modified container runtime process generates, on the first machine, a shim process representing the instantiated container image. The shim process forwards the instruction to an agent executing on a second machine, via a proxy connected to the agent via a network connection. The agent directs instantiation of the container image as a containerized process. A scheduler component executing on the first machine determines to migrate the containerized process to a third machine. The scheduler component directs migration of the containerized process to the third machine, during execution of the containerized process.

A method in one embodiment comprises implementing a kernel-based isolation layer, configuring a software container on the kernel-based isolation layer to include a dedicated operating system kernel as a library operating system, and executing one or more user processes in the software container. The method is performed by a cloud-based processing platform, an enterprise processing platform, or other type of processing platform comprising a plurality of processing devices, with each such processing device comprising a processor coupled to a memory. The library operating system illustratively runs in the software container at a privilege level that is the same as a privilege level of the one or more user processes executing in the software container. The library operating system is illustratively configured to support automatic translation of binaries of the one or more user processes in conjunction with converting system calls into corresponding function calls.

Method and system for improving software container performance and isolation

An apparatus comprises a first processing node configured to participate in a consensus protocol with a plurality of additional processing nodes. The first processing node is further configured in conjunction with its participation in the consensus protocol to implement repeated polling of respective selected subsets of the additional processing nodes, to resolve a state for a given transaction to a particular one of a plurality of possible states for the given transaction responsive to results of the repeated polling; and to initiate at least one automated action based at least in part on the resolved state for the given transaction. In some embodiments, the first processing node utilizes the results of the repeated polling to maintain a directed acyclic graph or other data structure of transactions that characterizes relationships between the given transaction and a plurality of other transactions.

Metastable byzantine agreement

In a method for improving the efficiency of a search engine in accessing, searching and retrieving information in the form of documents stored in document or content repositories, the search engine comprises an array of search nodes hosted on one or more servers. An index of the stored document is created. The search engine processes a user search query and returns a result set of query-matching documents. The index of the search engine is configured on the basis of one or more document properties and partitioned, replicated and distributed over the array of the search nodes. The search queries are processed on the basis of the distributed index. The method realizes a framework for distributing the index of a search engine across several hosts in a computer cluster, relying on three orthogonal mechanisms for index distribution, namely index partitioning, index replication, and assignment of replicas to hosts. In this manner, different ways of configuring the index of a search engine are obtained and provide a much improved resource usage and performance, combined with any desired level of fault tolerance.

Method for improving search engine efficiency

A scalable multicast protocol buffers the multicast messages at a subset of &ldquo;C&rdquo; members, where C is selected to reduce to an acceptable level the probability that a given message will be lost before it reaches at least one of the C members. When a member receives a multicast message, the member determines whether or not it should buffer the message by manipulating a string of bytes that is unique to both the message and the member and determining if the result is less than a calculated value C/n, where &ldquo;n&rdquo; is the number of known members. When one of the C bufferers thereafter receives a gossip message that indicates that the multicast message has been lost to the gossiping member, the bufferer retransmits the message to the gossiping member. When a member that is not one of the C bufferers receives such a gossip message, the member determines which members are bufferers of the lost message and requests that one of the bufferers retransmit the message to the gossiping member. The selected member identifies the bufferers by manipulating the byte strings associated with the lost message and the respective members that are known to the selected member. The selected member then sends to one of the identified bufferers a request for retransmission of the lost message to the gossiping member. The multicast protocol may further include a mechanism to detect catastrophic failures in the multicast transmission. Each member includes the buffer discussed above and a relatively small, fixed-size &ldquo;short-term&rdquo; buffer that holds a limited number of the received messages in the order in which the messages are received. A member monitors any holes or gaps in the sequences of incoming messages, and detects a catastrophic failure when a received gossip message identifies one of the same holes or gaps. When such a failure is detected, the member sends a request for multicast retransmission of the associated missing message to the sender, and the sender multicasts the message to the group from the short term buffer.

Multicast protocol with reduced buffering requirements

A computer network resource monitoring and locating system includes one or more hierarchical management information bases (HMIBs) through which a user can locate or obtain information relevant to managing or locating various network resources. The system maintains a portion of the HMIB information on each node, and a user starts from a local node and navigates through the system using the contact information that is part of the HMIBs. Each network node provides information directly to an associated lowest level MIB, referred to herein as a group MIB, and maintains a copy of that MIB. The system condenses the information in the gossip MIB and provides the condensed information as a row of a next highest level MIB, which is referred to herein as a subnetwork MIB. The system further condenses the information in this MIB to produce a row of a next highest level MIB, and so forth. Each row of the MIBs includes an identifier that identifies the source of the information, and contact information for at least a non-empty subset of the associated nodes. Each node maintains a view of each group MIB to which it has supplied information. The node also maintains views of each of the higher level MIBs that link the node to the highest or root level MIB, which contains condensed information of interest about every participating node. A user can obtain, through the contact information in the root MIBs maintained by the nodes, a point of contact for more detailed information about any participating node, such as, for example, the nodes in other subnetworks, and so forth. The system uses a hierarchical gossiping scheme to send gossip messages through contact nodes of various levels, to ensure that each node uses up-to-date information. Each node may also send broadcast and non-broadcast gossip messages to other nodes in the network. The gossiped information thus propagates through the network, using parallel routes and including in the message uncondensed information about only a portion of the nodes and condensed information about the remainder of the nodes in the network.

Scalable computer network resource monitoring and location system

A replicated management information base (RMIB), or replicated spreadsheet, is maintained at nodes to share information among the nodes. The RMIB includes columns, or categories, of the information to be shared and rows for the respective &ldquo;member&rdquo; nodes. Each member node is responsible for the contents of its own row. When the node updates its row, the node sends out an update message to the other members. As the update message propagates through the membership, each member updates the row in its copy, or view, of the RMIB. The nodes may include version numbers in the update messages, so that the receiving node ignores older messages or applies the messages in order. The member nodes may gossip among themselves, to ensure that each node receives the most recent updates. Also, the nodes may monitor their respective network interfaces and send as updates any changes to the status of the interfaces. The nodes may also include the interface status information in the gossip messages, such that the receiving nodes send update information to active interfaces. The nodes may select nodes on each sub-network and send the update messages to the selected nodes. The selected nodes then broadcast the update messages to the members on their respective sub-networks.

Distributed processing system with replicated management information base

The Protocol Accelerator is a system which significantly reduces the communication latency of complex layered communication protocols. In particular, the protocol accelerator reduces both the message header overhead imposed by layered protocols and the message processing overhead by employing such techniques as classifying, collecting and aligning the headers, using shorter identifiers to represent information in immutable fields of the headers, introducing pre- and post-processing of a message, packet filtering, and packing and unpacking of messages in cases where a backlog of messages has to be processed by the protocol accelerator.

Growing community of inventors

Ithaca, NY, United States of America

Robbert Van Renesse