Emc IP Holding Company LLC

Emc Corporation

Zhanwei Wang

Tao Ma

Lei Chang

Lirong Jian

Lili Ma

Jeffrey Ira Cohen

Caleb E Welton

Hitoshi Harada

Radhika Reddy

Luke Lonergan

Gavin Sherry

Kurt Harriman

A method and system for executing database queries in parallel using a shared metadata store. The metadata store may reside on a master node, and the master node may distribute this metadata with query plans to worker nodes. The worker nodes may request additional metadata from the master node when necessary.

Parallel processing database system with a shared metadata store

A method, article of manufacture, and apparatus for managing data. In some embodiments, this includes an initial instruction for a file stored in a first storage system, determining that the initial instruction is not supported by the first storage system, identifying a combination of instructions to the first storage system after determining that the initial instruction is not supported by the first storage system, wherein the combination of instructions is based on the initial instruction, performing the identified combination of instructions on the file stored in the first storage system, and storing results of the performed identified combination of instructions.

Pluggable storage system for distributed file systems

A method and system for executing database queries in parallel using a shared metadata store. The metadata store may reside on a master node, where the master node is the root node in a tree. The master node may distribute query plans and query metadata to other nodes in the cluster. These additional nodes may request additional metadata from each other or the master nodes as necessary.

Parallel processing database system

A method and system for executing a query in parallel is disclosed. A master node may receive a query from a client and develop query plans from that query. The query plans may be forwarded to worker nodes for execution, and each query plan may be accompanied by query metadata. The metadata may be stored in a catalog on the master node.

Processing a database query using a shared metadata store

Method of providing access controls and permissions over relational data stored in a hadoop file system

A query is executed in a massively parallel processing data storage system comprising a master node communicating with a cluster of multiple segments that access data in distributed storage by producing a self-described query plan at the master node that incorporates changeable metadata and information needed to execute the self-described query plan on the segments, and that incorporates references to obtain static metadata and information for functions and operators of the query plan from metadata stores on the segments. The distributed storage may be the Hadoop distributed file system, and the query plan may be a full function SQL query plan.

Self-described query execution in a massively parallel SQL execution engine

Data location-based query optimization for scan operators in a large-scale distributed database system is disclosed. In various embodiments, a set of query processing segments to be used to process a query is identified. It is determined for each of a plurality of data blocks to be scanned to process the query a corresponding storage location in which the data block is stored in a distributed storage system. Each of the plurality of data blocks is assigned to be scanned by a query processing segment selected based at least in part on a location of the query processing segment relative to the corresponding storage location in which the data block is stored.

Data locality based query optimization for scan operators

M&#xd7;N dispatching in a large scale distributed system is disclosed. In various embodiments, a query is received. A query plan is generated to perform the query. A subset of query processing segments is selected, from a set of available query processing segments, to perform an assigned portion of the query plan. An assignment to perform the assigned portion of the query plan is dispatched to the selected subset of query processing segments.

MxN dispatching in large scale distributed system

A system for sending a command stream comprises a first input interface, a processor, and an output interface. The first input interface is configured to receive a call. The processor is configured to determine one or more packets for the call. The one or more packets are able to be reassembled to form the call. The processor is configured to assign a first packet of the one or more packets to a first output queue and assign the second packet to a second output queue in the event there is a second packet of the one or more packets for the call. The output interface configured to send the first packet from the first output queue.

Supporting millions of parallel light weight data streams in a distributed system

A communication technique which includes determining, at least in part by comparing data associated with a packet that has been pulled from a received packet queue with a highest sequence number among packets that have been placed in the received packet queue, that the received packet queue has space available to receive a further packet. A receiver with which the received packet queue is associated is sent, based at least in part on the determination, a next packet.

Growing community of inventors

Beijing, China

Zhanwei Wang