Google Inc.

Hewlett-packard Development Company, L.p.

Hewlett Packard Enterprise Development LP

Yasushi Saito

Sanjay Ghemawat

Jeffrey Adgate Dean

Andrew Fikes

Arif Abdulhusein Merchant

Christos Karamanolis

Alistair Veitch

Sebastian Kanthak

Alexander Lloyd

Wilson Cheng-Yi Hsieh

Michael James Boyer Epstein

Craig A Soules

Michal Piotr Szymaniak

John Wilkes

Sean Quinlan

Svend Frolund

David Lowell

Susan Spence

Christopher Scott Taylor

Chris Jorgen Taylor

Christopher Cunningham Frost

A method for deleting obsolete files from a file system is provided. The method includes receiving a request to delete a reference to a first target file of a plurality of target files stored in a file system, the first target file having a first target file name. A first reference file whose file name includes the first target file name is identified. The first reference file is deleted from the file system. The method further includes determining whether the file system includes at least one reference file, distinct from the first reference file, whose file name includes the first target file name. In accordance with a determination that the file system does not include the at least one reference file, the first target file is deleted from the file system.

Method and system for deleting obsolete files from a file system

A method of accessing data includes storing a table that includes a plurality of tablets corresponding to distinct non-overlapping table portions. Respective pluralities of tablet access objects and application objects are stored in a plurality of servers. A distinct application object and distinct tablet are associated with each tablet access object. Each application object corresponds to a distinct instantiation of an application associated with the table. The tablet access objects and associated application objects are redistributed among the servers in accordance with a first load-balancing criterion. A first request directed to a respective tablet is received from a client. In response, the tablet access object associated with the respective tablet is used to perform a data access operation on the respective tablet, and the application object associated with the respective tablet is used to perform an additional computational operation to produce a result to be returned to the client.

Associating application-specific methods with tables used for data storage

A system, computer-readable storage medium storing at least one program, and a computer-implemented method for identifying a storage group in a distributed storage system into which data is to be stored is presented. A data structure including information relating to storage groups in a distributed storage system is maintained, where a respective entry in the data structure for a respective storage group includes placement metrics for the respective storage group. A request to identify a storage group into which data is to be stored is received from a computer system. The data structure is used to determine an identifier for a storage group whose placement metrics satisfy a selection criterion. The identifier for the storage group whose placement metrics satisfy the selection criterion is returned to the computer system.

Storing and moving data in a distributed storage system

A computer system issues a batch read operation to a tablet in a first replication group in a distributed database and obtains a most recent version of data items in the tablet that have a timestamp no great than a snapshot timestamp T. For each data item in the one tablet, the computer system determines whether the data item has a move-in timestamp less than or equal to the snapshot timestamp T, which is less than a move-out timestamp, and whether the data item has a creation timestamp less than the snapshot timestamp T, which is less than or equal to a deletion timestamp. If the determination is true, the computer system determines whether the move-out timestamp has an actual associated value and, if so, the computer system determines a second tablet in a second replication group in the database that includes the data item and issues the snapshot read operation to the second table in the second replication group to obtain a most-recent version of the data item that has a timestamp no greater than the snapshot timestamp T; otherwise, the computer system issues the snapshot read to the one tablet to obtain a most recent version of the data item that has a timestamp no greater than the snapshot timestamp T.

Efficient snapshot read of a database in a distributed storage system

A method for deleting obsolete files from a file system is provided. The method includes: receiving a request to delete a reference to a target file in a file system from a file reference data structure, wherein the file reference data structure includes target file names and reference file names; identifying a reference file name in the file reference data structure, wherein the reference file name includes a file name of the target file; deleting a reference file from the file system, wherein the reference file has the identified reference file name; checking whether the file system includes at least one reference file whose file name matches the file name of the target file; if there is no such reference file in the file system: deleting the target file from the file system; and deleting the file name of the target file from the file reference data structure.

A distributed storage system is provided. The distributed storage system includes multiple front-end servers and zones for managing data for clients. Data within the distributed storage system is associated with a plurality of accounts and divided into a plurality of groups, each group including a plurality of splits, each split being associated with a respective account, and each group having multiple tablets and each tablet managed by a respective tablet server of the distributed storage system. Data associated with different accounts may be replicated within the distributed storage system using different data replication policies. There is no limit to the amount of data for an account by adding new splits to the distributed storage system. In response to a client request for a particular account's data, a front-end server communicates such request to a particular zone that has the client-requested data and returns the client-requested data to the requesting client.

Organizing data in a distributed storage system

In one embodiment, the invention provides a method and apparatus for a wide area file system, including: creating a peer-to-peer wide-area file system that allows read and write sharing of data. In another embodiment, the invention provides a method and apparatus for a wide area file system, including using per-file replication to achieve high availability and performance in the wide-area distributed file system.

Symbiotic wide-area file system and method

The invention is a system for and a method of maintaining namespace consistency in a wide-area file system. In one embodiment, a wide-area file system has a plurality of replicas for a file. Each replica of a file and parent directories for the file are at a node. An update to a replica of the file is propagated to other replicas of the file. In response to receiving a propagated update to a replica at a node, the replica for the file at the node is updated. A directory operation (e.g., rename, link or unlink) may affect a backpointer for a replica. When a backpointer for a replica at a node is not consistent with the parent directories for the replica at the node, the the parent directories are modified to be consistent with the backpointer. A modification may be performed at a node while an earlier inconsistent modification may be ignored. As a result, consistency is maintained among the replicas.

Growing community of inventors

Mountain View, CA, United States of America

Yasushi Saito