Pivotal Software, Inc.

Emc Corporation

Emc IP Holding Company LLC

Gavin Sherry

Caleb E Welton

Tao Ma

Lei Chang

Zhanwei Wang

Lirong Jian

Lili Ma

Radhika Reddy

Florian Michael Waas

Jeffrey Ira Cohen

Hitoshi Harada

John Glenn Eshleman

Joy Jie Kent

Brian C Hagenbuch

Ashwin Kumar

Chris Pedrotti

A method, article of manufacture, and apparatus for managing data. In some embodiments, this includes an initial instruction for a file stored in a first storage system, determining that the initial instruction is not supported by the first storage system, identifying a combination of instructions to the first storage system after determining that the initial instruction is not supported by the first storage system, wherein the combination of instructions is based on the initial instruction, performing the identified combination of instructions on the file stored in the first storage system, and storing results of the performed identified combination of instructions.

Pluggable storage system for distributed file systems

A specified data type and pre-storage operation are receive. In response, an association between the two is created in storage metadata. After the association is created, data to be stored is received where the data has a data type. In response to receiving the data, the storage metadata, including the association, is accessed and it is determined if the data type of the data is the same as the specified one. If so, the specified pre-storage operation is automatically performed on the data in order to obtain an output and the output is stored.

Declaring directives in a data store

The invention identifies and caches query plans in a shared-nothing distributed data store that are unlikely to become invalid because they do not reference objects that are likely to be changed or deleted. Plans that are likely to become invalid and are not cached are those plans that reference data that is partitioned across segment/query execution nodes of the data store, plans that are complex, and plans that reference objects that are not 'built-in' (primitive) objects. The effect is that most plans which are generated on a query dispatch (master) node are not cached, whereas most plans generated on an execution (segment) node are cached.

Query plan management in shared distributed data stores

An association between a set of data and a pre-storage operation is obtained where the set of data is defined in a manner that permits the set of data to include more than one unit of data. In response to receiving data which meets the definition of the set of data, the pre-storage operation is performed on the received data and the output of the pre-storage operation is stored.

Data conversion algorithms such as compression and decompression algorithms or encryption and decryption algorithms are registered in a shared data store for selective use on data by creating implementations of a plurality of functions that as a group characterize the algorithms, converting the implementations to executable code and storing the code in a shared library, and creating and registering data store objects in a data store catalog with a common identifier that enables the algorithm to be selectively called and applied to data.

Registration and utilization of arbitrary data conversion schemes in a shared data store

A computer readable storage medium includes executable instructions to evaluate an expanded shared-nothing data store configuration. A data redistribution schedule table with specified parameters is formed. Data is redistributed within the expanded shared-nothing data store in accordance with the data redistribution schedule table.

Apparatus and method for expanding a shared-nothing system

Inverse distribution operations are performed on a large distributed parallel database comprising a plurality of distributed data segments to determine a data value at a predetermined percentile of a sorted dataset formed on one segment. Data elements from across the segments may be first grouped, either by partitioning keys or by hashing, the groups are sorted into a predetermined order, and data values corresponding to the desired percentile are picked up at a row location of the corresponding data element of each group. For a global dataset that is spread across the database segments, a local sort of data elements is performed on each segment, and the data elements from the local sorts are streamed in overall sorted order to one segment to form the sorted dataset.

Inverse distribution function operations in a parallel relational database

Updatable cursors for positioned updates are provided in a distributed shared-nothing data store by associating with each row of data in a result set hidden metadata that identifies the segments and the locations on such segments of such row. When an update is issued to data under a cursor, the associated metadata is transparently inserted into the update statement before it is dispatched to the segments to insure that the correct data is accurately updated. For cached query statements, generic metadata identifiers are used for building and caching, and the actual metadata identifiers are folded into the statement in place of the generic identifiers before dispatch.

Positioned updates in a distributed shared-nothing data store

A method for upgrading software in a shared data store comprises a reliable, safe in-place method of transformation of metadata in which a metadata catalog is cloned, stored in the data store, and the cloned metadata is transformed in-place using the data store engine and SQL declarative language. The original and transformed cloned catalog files are swapped atomically, and the system upgrade is validated using the new version of software. If errors are encountered during the upgrade, the original catalog can be swapped back and the system returned to its original state.

Growing community of inventors

San Mateo, CA, United States of America

Gavin Sherry