Storage and retrieval of filesystem metadata

Abstract

Methods, systems, and devices for data management are described. A data management system (DMS) may be employed to manage data associated with one or more computing systems. The DMS may generate metadata when capturing snapshots, which may contain information corresponding to files that have been created, deleted, modified, or otherwise changed since a prior snapshot. The DMS may use learning techniques and experimentation to perform data engineering to analyze metadata such as to perform anomaly detection. The DMS may implement techniques for efficiently storing and organizing metadata for use with machine learning processes—e.g., to support efficiently extracting datasets useful for machine learning experimentation. To obtain a dataset, the DMS may read metadata from different partitions of a storage environment in parallel, and accordingly, metadata may be stored in separate partitions based on customers and time ranges (e.g., to support such parallel reading).