Hewlett Packard Enterprise Development LP

International Business Machines Corporation

Globalfoundries U.S. Inc.

Suparna Bhattacharya

Douglas L Voigt

Samuel Scott Adams

Kalapriya Kannan

James R Kraemer

Robert R Friedlander

Paul Edward McKenney

Igor Arsovski

Lin A Nease

Arvind Kumar

Sonu Sudhakaran

Madhumita Bharde

Ted Dunning

Glyn Bowden

Janice M Zdankus

Vamsi Krishna Sangavarapu

Inderpal Singh Narang

Richard John Moore

Karen Wolfe Brannon

Muthukumar Murugan

Annmary Justine Koomthanam

Neeraj Gokhale

John Maxwell Cohn

Peter Frank Corbett

Paolo Faraboschi

Bruce Rabe

Martin Foltin

Hui-I Hsiao

Krishnan Santhana Rengarajan

Gary Paul Noble

Boris Zuckerman

John Michael Czerkowicz

Sandya Srivilliputtur Mannarswamy

Sergey Serebryakov

Maneesh Soni

Aalap Tripathy

Santigopal Mondal

Priya Pappanaickenpalayam Muthuswamy

David John Kleikamp

Mingming Cao

Priyanka Sood

Prashanto Kochavara

Dileep Deepa Shree

Chaitra Kallianpur

Ancy Sarah Tom

Krishnaraju Thangaraju

Theodore Y T Tso

Anand Krishnamoorthy

Douglas L Voight

Examples described herein relate to preparing datasets in a storage device for machine learning (ML) applications. Examples include maintaining ML facet mappings between ML facets and dataset preparation tags, deriving ML facets of a dataset stored in the storage device, and generating filtered datasets from the datasets using the ML facets and ML facet mappings. The filtered dataset is associated with improved dataset quality compared to unfiltered dataset. The storage device transmits the filtered dataset to ML applications requesting the dataset. Some examples include recommending, by the storage device, ML facets to the ML application based on performance metrics.

Machine learning facets for dataset preparation in storage devices

Systems and methods are provide for automatically constructing data lineage representations for distributed data processing pipelines. These data lineage representations (which are constructed and stored in a central repository shared by the multiple data processing sites) can be used to among other things, clone the distributed data processing pipeline for quality assurance or debugging purposes. Examples of the presently disclosed technology are able to construct data lineage representations for distributed data processing pipelines by (1) generating a hash content value for universally identifying each data artifact of the distributed data processing pipeline across the multiple processing stages/processing sites of the distributed data processing pipeline; and (2) creating an data processing pipeline abstraction hierarchy for associating each data artifact to input and output events for given executions of given data processing stages (performed by the multiple data processing sites).

Method to track and clone data artifacts associated with distributed data processing pipelines

Systems and methods are provided for reusing machine learning models. For example, the applicability of prior models may be compared using one or more assessment values, including a similarity threshold and/or an accuracy threshold. The similarity threshold may identify a similarity of data between a first data set used to generate a first model and a new data set that is received by the system. When the similarity between these two data sets is exceeded, the system may reuse a model with the highest similarity value. When an accuracy value of the data set does not exceed an accuracy threshold, the system may initiate a retraining process to generate a second ML model associated with the second data.

Artificial intelligence optimization platform

Systems and methods provide a system that gathers information about data as it progresses through data processing pipelines of data analysis projects. The data analytics system derives value indicators and implicit metadata from the data processing pipelines. For example, the data analytics system may derive value indicators and implicit metadata from data-related products themselves, semantic analysis of the code/processing steps used to process the data-related products, the structure of data processing pipelines, and human behavior related to production and usage of data-related products. Once a new data analysis project is initiated, the data analytics system gathers parameters and characteristics about the new data analysis project and references the value indicators and implicit metadata to recommend useful processing steps, datasets, and/or other data-related products for the new data analysis project.

Data recommender using lineage to propagate value indicators

Examples include bypassing a portion of an analytics workflow. In some examples, execution of an analytics workflow may be monitored upon receipt of a raw data and the execution may be interrupted at an optimal bypass stage to obtain insights data from the raw data. A similarity analysis may be performed to compare the insights data to a stored insights data in an insights data repository. Based, at least in part, on a determination of similarity, a bypass operation may be performed to bypass a remainder of the analytics workflow.

Similarity analyses in analytics workflows

Example implementations relate to partial file system instances. In an example, a subset of objects of a source file system instance on a source system are replicated to a target system to form a partial file system instance on the target system comprised of the subset of objects. Each of the objects of the source file system instance is identified by a signature based on content of each of the objects and the objects exhibit a hierarchical relationship to a root object in the file system instance. An unmaterialized object is dynamically added to the partial file system instance by replicating the corresponding object from the source file system instance. The target system is asynchronously updated from the source file system instance based on a comparison of the partial file system instance to the source file system instance.

Partial file system instances

Architectures and techniques for providing persistent volume functionality are disclosed. A storage container having a virtual storage volume to be persisted across multiple applications is created. The multiple applications hosted in one or more application containers. The storage container is placed within a virtual machine object. The virtual machine object containing the storage container is stored in a computer-readable memory as a persistent virtual storage volume.

Persistent volume plugin for containers

A source system generates snapshots of collected data. The snapshots have respective associated time references. Responsive to a request from a target system for data collected over a time interval, the source system generates a subset of the data collected by determining a start snapshot and an end snapshot. The start snapshot and the end snapshot are determined as a pair of snapshots that have respective associated time references that are most closely spaced and are inclusive of the time interval. The source system determines a difference in the data included in the end snapshot and the start snapshot and provides the subset of the data as the difference in the data included in the end snapshot and the start snapshot.

Data transfer using snapshot differencing from edge system to core system

An example implementation may relate to an apparatus that may identify data content of interest from data in buffers, and may store index entries representing the identified data content in a hierarchical index having different performance levels. The apparatus may include a priority manager that maintains an index scoreboard that tracks where index entries are to be stored among the different performance levels of the hierarchical index based on predetermined polices that prioritize data content of interest or functions that use data content of interest.

Hierarchical index involving prioritization of data content of interest

Ternary content addressable memory (TCAM) structures and methods of use are disclosed. The memory architecture includes one or more ternary content addressable memory (TCAM) fields, and control logic that applies progressively discriminating data-masking and scores a closeness of a match based on matched and mismatched bits.

Ternary content addressable memory

Example techniques of data management in a network environment are described. In an example, a semantic pattern in a data stream transmitted from a source device to an edge device in the network environment is determined. The semantic pattern indicates relevance of data samples in the data stream for analysis of the data stream. The data stream is processed based on the semantic pattern, for storage and transmission in the network environment.

Data management in a network environment

Some examples relate to associating an insight with data. In an example, data may be received. A determination may be made that data type of the data is same as compared to an earlier data. An insight generated from the earlier data may be identified, wherein the insight may represent intermediate or resultant data generated upon processing of the earlier data by an analytics function, and wherein during generation metadata is associated with the insight. An analytics function used for generating the insight may be identified.

Associating insights with data

Growing community of inventors

Bengaluru, India

Suparna Bhattacharya