International Business Machines Corporation

Subashini Balachandran

Rui Zhang

Wayne A Sawdon

Mihail Corneliu Constantinescu

Lukas Rupprecht

Frank Bernhard Schmuck

John Thomas Olson

Renu Tewari

Hongzhi Wang

Owen Theodore Anderson

Jan Hendrik Pieper

Provided are a computer program product, integrated cache manager, and method for maintaining data in a first level memory and buckets representing regions of memory devices to extend data cache. A plurality of buckets represent distinct regions of memory devices. The buckets are associated with different threshold access count ranges. Data having an access count is stored in one of the buckets associated with a threshold access count range including the access count of the data to store. Data evicted from a first level memory is copied to an initial bucket comprising one of the buckets. Data is moved from a source bucket comprising one of the buckets, including the initial bucket, to a target bucket of the buckets having a target threshold access count range including an access count of the data to move.

Maintaining data in a first level memory and buckets representing regions of memory devices to extend data cache

Provided are a computer program product, system, and method for using and training a machine learning module to determine actions to be taken in response to file system events in a file system. A file system event is detected. An action to be performed corresponding to the file system event is selected from an action list. A determination is made as to whether an outcome in the computing system resulting from the performed action satisfies an outcome threshold. A machine learning module is trained to increase a likelihood of selecting the performed action corresponding to the file system event when the outcome satisfies the outcome threshold. The machine learning module is trained to decrease a likelihood of selecting the performed action corresponding to the file system event when the outcome does not satisfy the outcome threshold.

Using and training a machine learning module to determine actions to be taken in response to file system events in a file system

Various embodiments are provided for managing memory for image analysis in a computing environment by a processor. An estimated amount of memory may be reserved for an image analysis job. During a processing of the image analysis job, at least one thread of the image analysis job is partially suspended by restricting the image analysis job from requesting allocation of additional memory upon memory requirements for the image analysis job exceeding the estimated amount of memory. Commensurate with partially suspending the at least one thread, a state of the image analysis job is maintained in the memory notwithstanding the processing of the at least one thread associated with the state is suspended.

Memory management for complex image analysis

In one embodiment, a method includes filtering a plurality of files stored to a central cluster of a distributed file system to place independent portions of the plurality of files into a plurality of groups using filters prior to receiving a query on the plurality of files. Files within each of the plurality of groups share a common searchable characteristic. The method also includes receiving, at the central cluster, an indication of the query. Moreover, the method includes responding to the query by duplicating files of one or more of the plurality of groups that correspond to the query to a local cluster of the distributed file system that provided the indication of the query and is geographically diverse from the central cluster. Other methods, systems, and computer program products for cognitive data management are described in accordance with more embodiments.

Cognitive file and object management for distributed storage environments

In one embodiment, a method for cognitively filtering data in a storage environment includes generating a plurality of cognitive data filters based on one or more factors to create a plurality of groups. The method also includes cognitively filtering data received by a global data repository to place independent portions of the data into the plurality of groups using the plurality of cognitive data filters prior to receiving a query on the global data repository. Data within each of the plurality of groups share a common definable characteristic. Other systems, methods, and computer program products for cognitively filtering data within storage environments are presented in accordance with more embodiments.

Cognitive data filtering for storage environments

A computer-implemented method according to one embodiment includes identifying an input/output (I/O) operation to be implemented within a distributed computing environment, where the distributed computing environment executes a plurality of different jobs, determining information associated with the I/O operation indicating that the I/O operation is associated with a recovery of one of the plurality of different jobs, and assigning an implementation priority to the I/O operation, based on the information associated with the I/O operation.

Prioritizing I/O operations

Embodiments relate to enabling sequential access for data scans on files with metadata. More specifically, object files are stored in a data storage. The objects are classified. A metaobject file is created in the data storage. The metaobject file is created by extracting attributes from the file from objects in a first classification. An entry is created in the metaobject file and the extracted attributes are stored in that entry. A file attribute is extracted from a file in the second classification and associated with the created entry in the metaobject file. A link is created between the file in the second classification and the related entry in the metaobject file.

Organization for efficient data analytics

The invention provides a method for reducing identification of chunk portions in data de-duplication. The method includes detecting sequences of stored identification of chunk portions of at least one data object, indexing the detected stored identification of chunk portions based on a sequence type, encoding first repeated sequences of the stored identifications with a first encoding, encoding second repeated sequences of the stored identifications with a second encoding, and avoiding repeated stored identifications of chunk portions.

Method and apparatus to minimize metadata in de-duplication

The invention provides a method and apparatus for determining sizing of chunk portions in data de-duplication. The method chunks input data into segments where each segment has a first size, assigns an identifier to each of the data segments, assigns an index to each of the identifiers, creates a suffix structure and a longest common prefix structure from the indexes, detects repeated sequences of indexes and non-repeated indexes from the suffix structure and the longest common prefix structure, determines a second size based on said detected repeated sequences and non-repeated indexes, and chunks the input data into a second plurality of data segments each having the second size.

Growing community of inventors

Sunnyvale, CA, United States of America

Subashini Balachandran