Emc IP Holding Company LLC

Dell Products Inc.

Lu Lei

Ao Sun

Gary Jialei Wu

Yu Teng

Julius Jian Zhu

Chen Xiao Wang

Ronnie Yu Cai

Wesley Wei Sun

Mikhail Danilov

Chun Xi Kenny Chen

Dong Wang

Pengfei Su

Jialei Wu

Yurun Wu

Xiao Ling Chen

Chao Chen

Kalyan Chakravarthy Gunda

Konstantin Buinov

Jingjing Liu

Shashwat Srivastav

Sriram Sankaran

Flavio Junqueira

Jian Zhu

Jun Luo

Igor Medvedev

Jian Gong

Grissom Tianqing Wang

Frank Yifan Huang

Jiang Cao

Xiao Le Shang

Patrick Minggang Lu

Alex Pengbo Zhang

Willa Lang Yuan

Aaron Yurun Wu

Young Yangchun Wu

Heyang Sun

Yong Que

Peter Jie Song

Nolan Shu Zhu

Sheng Ni

Raúl Gracia Tinedo

Xiaoxiao Mao

Shu Jiang

Lemonie Mengchi Li

The technologies described herein are generally directed toward writing multiple data objects to object storage. An example method can include receiving an aggregated data object comprising a set of data objects and metadata corresponding to respective data objects of the set of data objects. The method can further include updating the metadata to comprise the respective storage locations. Further, the method can include updating the metadata to include the respective storage locations. The method can further include allocating the metadata to be stored by metadata processing equipment, resulting in the metadata being stored in metadata storage equipment.

Writing multiple data objects to object storage

The technologies described herein are generally directed toward retrieving data from streaming storage. In an embodiment, a method can include receiving an application data request that identifies application data to be retrieved from a sequence of stored data chunks that correspond to a stored stream of data. The method can further include, based on the application data request, estimating a first estimated location of the application data, with the first estimated location including an identified chunk of a sequence of chunks. Further, the method can include, based on the application data request and a characteristic of the identified chunk, retrieving, by the system, a first data block that is estimated to comprise the application data, resulting in a first retrieved data block.

Retrieving data from streaming storage

A system of managing data, comprising a streaming data storage system operating on a processor that causes the processor to receive streaming data and to process the streaming data in response to an append command, a long-term storage system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate a chunk of data, an object storage system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate an object of data and a tables system operating on a processor that causes the processor to receive the processed streaming data from the streaming data storage system and to generate a table of data.

Formatted long-term storage of streaming system to allow data access through both stream and object API

Indices of storage systems are managed. An example method includes: receiving a flush cycle for flushing expired events in a storage system including multiple events in a data stream, determining, based on the flush cycle, a time slice for managing the index of the storage system, creating a slice index node for the time slice in response to determining that the occurrence time of a first event that enters the storage system in the data stream is within the time slice, wherein the slice index node includes an index node of the first event, and adding the slice index node to the index. A corresponding device and a corresponding computer program product are provided. Thus, the index of a large number of events entering the storage system is managed according to the occurrence time of the events, and the storage system can be searched and updated accurately and effectively.

Method, device, and program product for managing index of storage system

Transmission of data for object storage, such as stream transmission for object storage, is disclosed. For instance, a group of objects acquired from an object layer is stored in a storage space for storing an object stream and serves as a first part of the object stream; and, in response to that an event related to at least one object in the group of objects occurs at a client terminal, the event-related information is stored in the storage space and is used as a second part of the object stream, the event-related information including at least one of the following: an identifier of the at least one object, a type of the event, and metadata of the event.

Method, device, and computer program product for transmitting data for object storage

Embodiments of the present disclosure relate to processing data. An example method includes acquiring data related to a first moment in streaming data of an object to be processed. The method further includes storing the data in a first entry of a data table based on an identification of the object to be processed, wherein the data table further includes a second entry before the first entry, and the second entry stores data related to a second moment before the first moment in the streaming data. The method further includes updating an index related to the object to be processed based on the first entry. Thus, a solution to the problem of performing search in data at different moments is provided, and it is unnecessary for a user to participate in the solution, thus improving the user experience and reducing the use of storage resources.

Method, electronic device, and computer program product for processing data

Techniques for processing an access request and updating a storage system are provided. For instance, a method comprises: receiving an access request for an object associated with a storage system, the storage system including a plurality of physical nodes, each of the plurality of physical nodes including at least one set of virtual units, each set of virtual units including at least one virtual unit; determining, from a plurality of sets of virtual units included in the plurality of physical nodes of the storage system, a target set of virtual units associated with the object; and determining, from the target set of virtual units, a target virtual unit corresponding to the object. With the technical solution of the present disclosure, not only a set of virtual units on a physical node may be easily split and merged, but also huge computing resources that need to be allocated may be saved, so better user experience may be brought about at a lower cost.

Methods, devices, and a computer program product for processing an access request and updating a storage system

Retention of events of an ordered event stream is disclosed. Expiration of events stored in a segment of an ordered event stream (OES) can be desirable. New events are added to a head of an OES segment, and pruning events from a tail of the OES segment can be valuable. Processing applications can register a processing scheme for a segment, e.g., at-least-once processing, exactly-once processing, etc., and can generate checkpoints indicating a degree of advancement in processing events of the segment. The ordered event stream can determine a cut point indicative of a progress point, that before which, events of an OES can be marked as ready for expiration. However, events that are marked for expiration can be retained to allow processing based on a checkpoint, e.g., expiration of the event can be refused until there is an assurance the event was read by the processing application.

Ordered event stream event retention

Data stream processing is performed. For instance, a method includes: in response to a query request on an input data stream, determining data labels for input data items in the input data stream, the data labels at least indicating a first matching relationship between the input data items and a first query condition of the query request; determining an output data stream corresponding to the input data stream, the output data stream including output data items corresponding to the input data items, and each output data item having a data label; and determining query information corresponding to the query request based on the output data stream. In this way, it is possible to respond to real-time query requests for data streams.

Method, electronic device, and computer program product for data stream processing

Embodiments of the present disclosure provide a method and device for storing data. The method comprises: generating a data block corresponding to data to be stored; aligning the data block to a boundary of a tracking unit of a predefined size for validating the data; and storing the aligned data block in at least one storage unit of a storage space, the at least one storage unit having an identical size. The method according to embodiments of the present disclosure can align the data block so as to minimize the waste of storage space and avoid the situation where the rest data cannot be validated due to disappearance of partial data.

Method and device for storing data

Storage blocks are managed. For instance, a set of write parameters and a set of deletion parameters are obtained related to a target storage block. In response to the set of write parameters matching the set of deletion parameters, a first data length is obtained for the target storage block, the first data length being determined in response to receiving a write request for the target storage block. Further, reclaim information is determined related to the target storage block based on the first data length and the set of deletion parameters. It is thus possible to reduce times of scanning the entire object table to determine whether there is an object referring to the storage block, thereby reducing the time consumed by the verification process and improving the reclaiming speed of storage blocks.

Determining reclaim information for a storage block based on data length and matching write and delete parameters

Embodiments of the present disclosure provide a method, a device and a computer program product for data processing. The method comprises in response to data associated with an object being stored in the storage location, generating an entry indicating a first association relationship between the data and a storage location. The method further comprises generating metadata of the object, the metadata indicating a second association relationship between the object and the entry. Further, the method comprises in response to the first association relationship between the data and the storage location being changed, updating the entry independent of the metadata. Embodiments of the present disclosure may avoid the huge cost caused by changing metadata multiple times and reduce the system load and the impacts to the front end load.

Method, device and computer program product for data processing

Storage management is enabled. An example method comprises: receiving an update request for a target object stored in a first storage block to write the updated target object into a second storage block different from the first storage block; determining a candidate object associated with the target object using a search tree, the search tree indicating a hierarchical relation among a plurality of objects, wherein a first node corresponding to the target object and a second node corresponding to the candidate object share a same index node in the search tree; determining whether the candidate object was updated during a past predetermined time period; and in response to the candidate object not being updated during the past predetermined time period, moving the candidate object from a third storage block into a fourth storage block different from the third storage block.

Method, device and computer program product for storage management

Embodiments of the present disclosure relate to a method, device and computer program product for data stream processing. The method comprises activating, in response to receiving a starting first predefined tag in a first input data stream from a first data source, a duplicate process of a process by duplicating a state of the process to the duplicate process. The method further comprises processing, by the duplicate process, only a data unit before an ending second predefined tag in a second input data stream from a second data source without interruption of processing of the first and second input data streams by the process. The second data source differs from the first data source. The method further comprises creating a snapshot of the duplicate process for recovery of the process due to failure in response to receiving the ending second predefined tag in the second input data stream.

Method, device and computer program product for data stream processing

Growing community of inventors

Shanghai, China

Lu Lei