Robin Systems, Inc.

Krishna Satyasai Yeddanapudi

Rafit Izhak-Ratzin

Haim Helman

Dhanashankar Venkatesan

Shravan Kumar Vallala

Christopher Alan Mildebrandt

Rao V Madduri

Gurmeet Singh

Robert S Cancilla

Rafit Izhak Ratzin

Alon Pelled

A system provides systems and methods for visualizing a computational process. A portion of attributes of a computational process may be displayed on a face of a three-dimensional object. In response to rotation of the object by a user, a different face of the object may be displayed and a different portion of the attributes displayed thereon. Each object may represent a status of a computational cluster, specifically a set of jobs executing on the cluster. A cluster may be selected and the jobs represented by means of graphical indicators arranges on a time axis and a performance axis. The size of a graphical indicator may indicate an amount of processing time consume by the corresponding job. A color of the graphical indicator may indicate a status. A portion of the graphical indicator may be visually distinguished from a remaining portion to indicate a completion percentage of the job.

System and method performing job management

Systems and methods are disclosed for aggregating data capable of diagnosing unique datacenter issues. Traffic statistic collection may be moved from intermediate, datacenter nodes to end hosts providing reports for aggregation and correlation with events at an analytic controller, uncovering implications for such events. To track metrics and/or diagnose datacenter issues not addressed in traffic statistics, information locally available to the end hosts may be combined and/or correlated with traffic statistics. Examples may involve information about: virtual and physical computing resources; a sub-cluster; an application and/or process utilized by a datacenter task; a task/job type; an implementation phase; an initiating user; a task priority; link utilization and/or other traffic statistics relative to the foregoing. Also, for efficiency purposes, the analytic controller may apply a hash to map a virtual to a physical IP address in determining a datacenter path based on topology limited to a physical network, saving computational expense.

Providing a data set for tracking and diagnosing datacenter issues

Systems and methods are disclosed for traffic engineering and traffic flow control in a datacenter. Flow agents may be provided at hosts that may be operable to control a rate at which a traffic flow is sent from one point to another within a datacenter to facilitate differing levels of Quality of Service (QoS). The differing levels of QoS may be implemented by differing sizes of bandwidth allocations assigned remotely by a traffic engineering controller. The traffic engineering controller may distribute the differing allocations of bandwidth to the flow agents corresponding to the traffic flows to keep the traffic flow rates within the bandwidth allocation. An augmenting function may also be applied at the traffic engineering controller to maximize bandwidth utilization by checking for unutilized bandwidth and reallocating such bandwidth in accordance with one or more QoS.

Fine-grained quality of service in datacenters through end-host control of traffic flow

Systems and methods are disclosed for generating traffic statistics for a datacenter. Distributed, parallel processing may be used to generate traffic statistics from data sets about traffic in a datacenter. To reduce data sets from which such statistics are derived to manageable sizes and relevant processing times for distributed, parallel processing, traffic agents may be provided at end hosts in the datacenter. The traffic agents may summarize data traffic over large numbers of packets in terms of the various sockets over which they are transmitted. Reports on the various sockets may be sent by the various flow agents that monitor them to an analytic controller. The analytic controller may aggregate, provide flow-path information for, further reduce, and/or provision the resultant data for distributed parallel processing.

Traffic statistic generation for datacenters

Systems and methods are disclosed for interacting with a file system. The file system is operable to reside in user space of a computing system. A module, also within user space, may provide a messaging service supporting requests from an application to the file system. By bypassing a System-Call Interface (SCI) of the computing system's kernel space, the module may support requests from the application to the file system with enhanced efficiency and/or customizable features not provided by the SCI. In some examples, the module may include a library in an independent layer within user space and below the application, allowing the library to provide an application-independent messaging service for different applications. Furthermore, in some examples, the module may include a segment of memory, within user space, shared between the application and the file system for passing data involved in requests and/or responses to and/or from the file system.

File-system requests supported in user space for enhanced efficiency

Systems and methods are disclosed for classifying traffic flows. A traffic agent operable to collect classification information for one or more traffic flows may be deployed at an end host communicatively coupled to a data-center network. The traffic agent, deployed in a user space independent of the operating system, may compare the classification information for a given traffic flow to a metric value. Where the classification information achieves a certain threshold indicated by the metric value, the traffic agent may classify the traffic flow as an elephant flow. In some examples, a library may be included with the traffic agent that may include a modified send function. The modified send function may provide classification information to the traffic agent indexed to the traffic flow for which it is called so that the traffic agent may analyze the classification information to potentially provide a classification for the traffic flow.

Traffic flow classification

Systems and methods are disclosed for scheduling jobs processed in a distributed fashion to realize unharnessed efficiencies latent in the characteristics of the jobs and distributed processing technologies. A job store may be communicatively coupled to a job analyzer. The job analyzer may be operable to read information characterizing a job to identify multiple data blocks to be processed during the job at multiple locations in a cluster of nodes. A scheduling module may use information about the multiple data blocks, their storage locations, their status with respect to being provisioned to processing logic, data blocks to be processed by other jobs, data blocks in cache that have been pre-fetched for a prior job, quality-of-services parameters, and/or job characteristics, such as job size, to schedule the job in relation to other jobs.

Asynchronous scheduling informed by job characteristics and anticipatory provisioning of data for real-time, parallel processing

Systems and methods are disclosed for reducing latency in processing data sets in a distributed fashion. A job-queue operable for queuing data-processing jobs run on multiple nodes in a cluster may be communicatively coupled to a job analyzer. The job analyzer may be operable to read the data-processing jobs and extract information characterizing those jobs in ways that facilitate identification of resources in the cluster serviceable to run the data-processing jobs and/or data to be processed during the running of those jobs. The job analyzer may also be coupled to a resource warmer operable to warm-up a portion of the cluster to be used to run a particular data-processing job prior to the running of the job. In some embodiments, mappers and/or reducers may be extracted from the jobs and converted into compute node identifiers and/or data units identifying blocks for processing, informing the warm-up operations of the resource warmer.

Anticipatory warm-up of cluster resources for jobs processed on multiple cluster nodes

In one exemplary aspect, a method of managing computer network traffic flow quality of service includes the step of configuring a configurable network device to provide a specified quality of service to a data packet with a specified quality of service configuration based on a quality of service classification port number in a data packet header of the data packet. At the source node, a data packet is generated. At the source node, replacing the destination port number in the data packet header with a quality of service classification port number. At the source node, the destination port number is included in an options field of the data packet's header. The data packet is communicated to the configurable network device. At the destination node, receiving the data packet, replacing the quality of service classification port number with the original destination port number, and forwarding the packet to a destination process.

Growing community of inventors

Pleasanton, CA, United States of America

Krishna Satyasai Yeddanapudi