Emc Corporation

Emc IP Holding Company LLC

John Forecast

Wayne William Duso

Sorin Faibish

Peter C Bixby

Uresh K Vahalia

Daniel Gardere

Philippe Armangau

Uday K Gupta

Dennis P Ting

Michel Noury

Jean-Pierre Bono

Percy Tzelnic

Frank Charles Paterra

John T Cardente

Dinesh Venkatesh

Seyfullah H Oguz

Zachary Merlynn Loafman

Deepak Sunny Veliath

John Michael Bent

Steven T McClure

Xiaoye Jiang

Sitaram D Pawar

Stephen A Fridella

David Reese

Frederic Corniquet

Vadim Longinov

Dennis P J Ting

Mukesh Gupta

Michael B Evans

Matthew H Fredette

Lev Vaitzblit

Serge Joseph Pirotte

Ugur Sezer

Steve Chalmer

Sebastian Keller

Raymond Mantchala

Jean Louis Rochette

Leslie E Kuczynski

Kannan M Alagappan

William O Hultin

Operating at least one hypervisor includes running a first hypervisor as a first thread of an underlying operating system, running a second hypervisor as a second thread of the underlying operating system, loading a first guest operating system using the first hypervisor based on the first thread of the underlying operating system, loading a second guest operating system using the second hypervisor based on the second thread of the underlying operating system, and scheduling sharing of resources of the underlying system between the first hypervisor and the second hypervisor according to a scheduler of the underlying operating system, where the first hypervisor and the second hypervisor run independently of each other. The scheduler of the underlying operating system may schedule fractional time shares for the first hypervisor and the second hypervisor to access the same resource.

Extended fractional symmetric multi-processing capabilities to guest operating systems

Various embodiments are directed toward improving computing performance for data intensive computing applications by arranging a physical machine into a single computing appliance that includes tightly coupled dedicated computation and storage resources that may scale to multiple peta-byte or greater storage in concert with providing a similarly scaling computing and data caching platform, monitoring co-resident guest applications, and data access applications. Generally, the computing appliance may be arranged to execute compute applications and data storage applications in a single self-contained computing device. The data storage applications may execute within virtual machines (VM's) and the compute applications may execute in other co-resident VM's residing one or more computing appliances.

Combined data storage and computing appliance that provides scalable storage in a clustered computing environment

A multi-core processor of a network attached storage system processes requests from host computers for services of a file system service. Each core maintains endpoints of respective connection-layer connections to the hosts to affine respective streams of network traffic with the core, and dynamically and preferentially assigns execution threads of the core to process file system service requests of the streams affined with the core. Each core also co-operates with the other cores to dynamically and non-preferentially (a) assign execution threads of the core to process file system service requests of the streams affined with the other cores, and (b) assign execution threads of the other cores to process file system service requests of the streams affined with the core, promoting efficient use of the cores for the processing workload of the file system service.

Multi-core storage processor assigning other cores to process requests of core-affined streams

A burst buffer appliance is adapted for coupling between a computer system and a file system. The burst buffer appliance comprises a first memory, at least one additional memory, and a controller. The controller is configured to detect a remote direct memory access operation between a given compute node of the computer system and the additional memory, and to execute the detected remote direct memory access operation in a manner that bypasses utilization of an operating system associated with the first memory. The first and additional memories may comprise respective main and flash memories. The controller may comprise a virtual machine arranged between a host channel adaptor of the burst buffer appliance and the additional memory, with the virtual machine configured to identify a remote direct memory access operation directed to an address space of the additional memory and to control the flow of data associated with that operation.

Burst buffer appliance with operating system bypass functionality to facilitate remote direct memory access

A virtual switch implemented in a NIC routes packets to a specific virtual machine of a group of supported virtual machines based on a unique MAC address assigned to that virtual machine. Packets which are addressed to the broadcast or a multicast address may be copied to all virtual machines associated with the virtual switch or may be passed to the NIC device driver so that it may be copied by software. The NIC operates in a selective mode whereby packets that are not addressed to a supported virtual machine are rejected or discarded. Further, the virtual switch routes the packets to the virtual machines by performing a DMA copy operation to dedicated memory associated with the virtual machine to which the packet is addressed. The NIC may also include dedicated memory for each supported MAC address in order to facilitate operation.

Direct memory access through virtual switch in device driver

Various embodiments are directed toward improving Virtual Machine performance by reducing the amount of communication protocol processing performed between co-resident Virtual Machine guest applications executing on the same physical machine. Overhead can be reduced by short-circuiting the full communication protocol stack when a plurality of Virtual Machine guest applications are hosted and executing on the same physical machine such as a data storage node or a compute accelerator node. Improvements may also be realized by enabling the sharing in virtual memory of physical memory pages between co-resident Virtual Machine guest applications, and employing a low latency high bandwidth network for enabling guest applications to communicate with each other that are located in different nodes in a data storage cluster.

Accelerating data intensive computing applications by reducing overhead associated with communication protocols

A device for providing a plurality of virtual machines utilizes a multi-core processor having a plurality of cores, each with a memory cache, and a shared memory resource in communication with the cores. The device utilizes a cooperative, multi-level scheduler. The multi-level scheduler includes a primary scheduler and a plurality of secondary schedulers, each supporting a subset of the physical cores. The primary scheduler assigns a group of threads to one of the processor cores. The secondary scheduler associated with the processor core to which the group of threads was assigned schedules execution of individual ones of the threads. The secondary scheduler also provides an indication of lock status to the primary scheduler. The lock status information can be used by the primary scheduler to avoid preempting a thread that holds a lock.

Cooperative multi-level scheduler for virtual engines

Based on a count of the number of dirty pages in a cache memory, the dirty pages are written from the cache memory to a storage array at a rate having a component proportional to the rate of change in the number of dirty pages in the cache memory. For example, a desired flush rate is computed by adding a first term to a second term. The first term is proportional to the rate of change in the number of dirty pages in the cache memory, and the second term is proportional to the number of dirty pages in the cache memory. The rate component has a smoothing effect on incoming I/O bursts and permits cache flushing to occur at a higher rate closer to the maximum storage array throughput without a significant detrimental impact on client application performance.

Rate proportional cache write-back in a storage server

Interrupts and code threads are assigned in a particular way to the core CPUs of a network file server in order to reduce latency for processing client requests for file access. Threads of the network stack are incorporated into real time threads that are scheduled by a real-time scheduler and executed exclusively by a plurality of the core CPUs that are not interrupted by disk adapter interrupts so that the disk adapter interrupts do not interrupt execution of the network stack. Instances of a storage access driver are hard affinity threads, and soft affinity threads include a multitude of instances of a thread of the file system stack for file access request processing so that file access request processing for a multitude of concurrent file access requests is load balanced over the core CPUs.

Servicing interrupts and scheduling code thread execution in a multi-CPU network file server

A file server system has a cluster of server computers that share access to a file system in shared storage. One of the server computers has primary responsibility for management of access to the file system. In order to reduce the possibility of primary server overload when a large number of the clients happen to concurrently access the same file system, most metadata processing operations are offloaded to secondary server computers. This also facilitates recovery from failure of a primary server computer since only a fraction of the ongoing metadata operations of a primary server computer is interrupted by a failure of the primary server computer. For example, a secondary data mover may truncate, delete, create, or rename a file in response to a client request.

Metadata offload for a file server cluster

Access to movies ranging from very popular movies to unpopular movies is managed by configuring sets of data movers for associated movie rankings, reserving data mover local cache resources for the most popular movies, reserving a certain number of streams for popular movies, negotiating with a client for selection of available movie titles during peak demand when resources are not available to start any freely-selected movie in disk storage, and managing disk bandwidth and primary and local cache memory and bandwidth resources for popular and unpopular movies. The assignment of resources to movie rankings may remain the same while the rankings of the movies are adjusted, for example, during off-peak hours. A movie locked in primary cache and providing a source for servicing a number of video streams may be demoted from primary cache to disk in favor of servicing one or more streams of a higher-ranking movie.

Video file server cache management using movie ratings for reservation of memory and bandwidth resources

A system for producing multiple concurrent real-time video streams from stored MPEG video clips includes a video server and at least one MPEG decoder array. The decoder array has multiple decoder pairs, each pair having a video switch for switching from one decoder in the pair to the other at a specified time. Switching may occur from a specified Out-point frame to a specified In-point frame, and the specified frames can be any frame type at any location in the group of pictures (GOP) structure. In a preferred construction, the video server has a controller server linked to a series of data mover computers, each controlling one or more respective decoder arrays. The data mover computers use a control protocol to control the decoder arrays, and each decoder uses a data protocol to request data from a respective data mover computer.

MPEG dual-channel decoder data and control protocols for real-time video streaming

Transferring data from at least one local data storage device to at least one remote data storage device includes subdividing the data into portions, assigning a sequence number to each of the portions, where writes to the at least one local storage device for a first set of portions having a first sequence number are begun before writes for a second set of portions having a second sequence number that is assigned after to the first sequence number, and updating the sequence number in response to a particular sequence number having been used before and there being data to send from the at least one local storage device to the at least one remote storage device. The at least one local storage device may include a plurality of storage entities that form a consistency group.

Spatial domain mechanism

Metadata for splicing of an encoded digital motion video stream (such as an MPEG Transport Stream) is prepared in real time while recording at the encoding bit rate and faster than encoded bit rate for off line encoding independent of the bit rate and mechanisms for ingestion of the data stream into data storage. Preprocessing is performed during a metered file transfer protocol (FTP) and includes pseudo real-time encoding. The preprocessing includes Group of Pictures (GOP) level pre-processing of splicing In Points and results in an intimate linkage between metadata and the file system in which the video data is stored. The preferred file system enables access to metadata in parallel to writing the data on disk. The pre-processing is performed simultaneous to writing the data to the disk using a carousel type buffer mechanism.

Preparation of metadata for splicing of encoded MPEG video and audio

Coded video from an on-line MPEG video encoder is stored as a clip in a video server or is otherwise received in the video server and prepared or used for splicing. In order to reduce apparent frame inaccuracy that may result from the splicing process, the on-line MPEG video encoder and the server are coordinated so that the group-of-picture (GOP) structure in the encoder provides specified In-points and Out-points that are valid and desirable for splicing. An encoder control protocol is also provided for remote control of the on-line MPEG video encoder in order to coordinate the on-line MPEG video encoder with the video server.

Growing community of inventors

Newton, MA, United States of America

John Forecast