Emc IP Holding Company LLC

Julius Jian Zhu

Lu Lei

Pengfei Su

Yu Teng

Mikhail Danilov

Jingjing Liu

Ao Sun

Igor Medvedev

Frank Yifan Huang

Lingling Yao

Alex Pengbo Zhang

Sheng Ni

Indices of storage systems are managed. An example method includes: receiving a flush cycle for flushing expired events in a storage system including multiple events in a data stream, determining, based on the flush cycle, a time slice for managing the index of the storage system, creating a slice index node for the time slice in response to determining that the occurrence time of a first event that enters the storage system in the data stream is within the time slice, wherein the slice index node includes an index node of the first event, and adding the slice index node to the index. A corresponding device and a corresponding computer program product are provided. Thus, the index of a large number of events entering the storage system is managed according to the occurrence time of the events, and the storage system can be searched and updated accurately and effectively.

Method, device, and program product for managing index of storage system

Transmission of data for object storage, such as stream transmission for object storage, is disclosed. For instance, a group of objects acquired from an object layer is stored in a storage space for storing an object stream and serves as a first part of the object stream; and, in response to that an event related to at least one object in the group of objects occurs at a client terminal, the event-related information is stored in the storage space and is used as a second part of the object stream, the event-related information including at least one of the following: an identifier of the at least one object, a type of the event, and metadata of the event.

Method, device, and computer program product for transmitting data for object storage

Embodiments of the present disclosure relate to processing data. An example method includes acquiring data related to a first moment in streaming data of an object to be processed. The method further includes storing the data in a first entry of a data table based on an identification of the object to be processed, wherein the data table further includes a second entry before the first entry, and the second entry stores data related to a second moment before the first moment in the streaming data. The method further includes updating an index related to the object to be processed based on the first entry. Thus, a solution to the problem of performing search in data at different moments is provided, and it is unnecessary for a user to participate in the solution, thus improving the user experience and reducing the use of storage resources.

Method, electronic device, and computer program product for processing data

Retention of events of an ordered event stream is disclosed. Expiration of events stored in a segment of an ordered event stream (OES) can be desirable. New events are added to a head of an OES segment, and pruning events from a tail of the OES segment can be valuable. Processing applications can register a processing scheme for a segment, e.g., at-least-once processing, exactly-once processing, etc., and can generate checkpoints indicating a degree of advancement in processing events of the segment. The ordered event stream can determine a cut point indicative of a progress point, that before which, events of an OES can be marked as ready for expiration. However, events that are marked for expiration can be retained to allow processing based on a checkpoint, e.g., expiration of the event can be refused until there is an assurance the event was read by the processing application.

Ordered event stream event retention

An index of a storage system is managed. For example, events in a data stream to be stored are received. According to a predetermined length of a time window and occurrence times of the events, an event among the events that occurs within the time window is determined. Based on the event, a window index node is created including an index of the event. In response to determining that a current time point meets a threshold time point corresponding to the time window, the window index node is added to the index, and the threshold time point indicates that the number of received events that occur within the time window in the data stream reaches a threshold number. Thus, an index can be created in time for a large number of events entering the storage system. Further, the storage system can be queried and updated accurately and effectively.

Data stream processing is performed. For instance, a method includes: in response to a query request on an input data stream, determining data labels for input data items in the input data stream, the data labels at least indicating a first matching relationship between the input data items and a first query condition of the query request; determining an output data stream corresponding to the input data stream, the output data stream including output data items corresponding to the input data items, and each output data item having a data label; and determining query information corresponding to the query request based on the output data stream. In this way, it is possible to respond to real-time query requests for data streams.

Method, electronic device, and computer program product for data stream processing

Storage blocks are managed. For instance, a set of write parameters and a set of deletion parameters are obtained related to a target storage block. In response to the set of write parameters matching the set of deletion parameters, a first data length is obtained for the target storage block, the first data length being determined in response to receiving a write request for the target storage block. Further, reclaim information is determined related to the target storage block based on the first data length and the set of deletion parameters. It is thus possible to reduce times of scanning the entire object table to determine whether there is an object referring to the storage block, thereby reducing the time consumed by the verification process and improving the reclaiming speed of storage blocks.

Determining reclaim information for a storage block based on data length and matching write and delete parameters

The present disclosure relates to a method, apparatus and computer program product for managing a storage system. According to implementations of the present disclosure, there is provided a method for managing a storage system. In the method, an object vector is created based on multiple objects stored in the storage system, here an object element in the object vector is associated with at least one object among the multiple objects. Multiple chunks included in the storage system are divided into multiple groups. For a group among the multiple groups, a chunk vector is created based on each chunk in the group, here a chunk element in the chunk vector is associated with a chunk in the group. A mapping matrix is created for managing the storage system based on the object vector and the chunk vector. Moreover, there is provided an apparatus and computer program product for managing a storage system. By means of the technical solution of the present disclosure, the number of storage units involved in processing and input/output time overheads may be reduced.

Growing community of inventors

Shanghai, China

Julius Jian Zhu