International Business Machines Corporation

Xiaomi Inc.

Beijing Xiaomi Mobile Software Co., Ltd.

Lenovo (beijing) Limited

Beijing Lenovo Software Ltd.

Beijing Jingdong Shangke Information Technology Co., Ltd.

Beijing Jingdong Century Trading Co., Ltd.

Lei Yu

Petr Novotny

Qi Zhang

Hong Min

Yuk Lung Chan

Fan Jing Meng

Jia Qi Li

Tian Wu

Nitin Gaur

Lin Y Yang

Zhi Shuai Han

Guosheng Li

Anyu Liu

Wei Li

Wei Chen

Ravi Nair

Zhenhua Zhang

Dakuo Wang

Shyam Ramji

Yi Zheng

Motohiro Kawahito

Takuya Nakaike

Lingfei Wu

Taesung Lee

Ke Shang

Xiang Chen

Chen Wang

Sean K Dunne

Xin Wei

Yanguang Liu

Yanjun Cui

Chengbin Liu

Hang Ba

Rong Shi

Jinhao Xu

Guilong Tan

Jun Dou

Abhishek Dokania

Gaofei Zhao

In accordance with an embodiment of the invention, a method is provided for performing data pre-processing operations during data preparation of a machine learning lifecycle. The method includes defining one or more data pre-processing functions for applying to data stored in a dataset, executing one or more learn functions for learning the data, and executing one or more transform functions for transforming the data. Each of the one or more learn functions generates a first Structured Query Language (SQL) statement representing a definition of corresponding learn function for corresponding defined data pre-processing function. Each of the one or more transform functions generates a second SQL statement representing a definition of corresponding transform function for corresponding defined data pre-processing function. The dataset is stored in a database.

Performing data pre-processing operations during data preparation of a machine learning lifecycle

An example operation may include one or more of generating a blockchain request which comprises data for a blockchain, creating a tag value which identifies whether the blockchain request is a noisy request that comprises fake data or a non-noisy request that comprises non-noisy data, storing the tag value within the blockchain request, and transmitting the blockchain request to one or more blockchain peers.

Noisy transaction for protection of data

A system may enact a process including initiating an operation to transfer a secret, establishing n groups of nodes in a blockchain network, dividing a secret into n secret parts, creating n group keys, encrypting each secret part of the secret with a corresponding group key, and recording encrypted secret parts on the blockchain networks.

Secret smart operations in blockchain

A computer-implemented method for visual question generation includes training an alignment module to analyze an image, an answer hint, and a visual hint with respect to the image. A k-nearest neighbors (KNN) graph is constructed by performing an aligned embedding for each region of the image. A node embedding component is generated by using a graph embedding component of the KNN graph. A visual question is generated by sequence decoding each image and graph of the image.

Visual question generation with answer-awareness and region-reference

An example operation may include one or more of receiving, via a first blockchain peer, a request to verify an identity of a user, performing, via the first blockchain peer, an identity validation of the user via a know your customer (KYC) service provider of a blockchain network, determining whether a digital identity of the user has been previously stored by another blockchain peer on a blockchain of the blockchain network, in response to a determination that the digital identity of the user has not been stored on the blockchain, creating a non-fungible token (NFT) that identifies the user, and appending information about the identity validation of the user to a structure of the NFT and storing the NFT in a block of the blockchain.

Token-based identity validation via blockchain

Mechanisms for performing intelligent federated machine learning (ML) model updates are provided. A plurality of ML model updates, and a plurality of dataset sketch commitment data structures (sketches), are received from a plurality of participant computing systems. Each sketch provides statistical characteristics of a corresponding local dataset used by a corresponding participant to train a local ML model. A potentially malicious participant identification operation is performed based on an analysis of the plurality of sketches to identify one or more potentially malicious participants based on differences in sketches. ML model updates received from participant computing systems identified as potentially malicious participants are discarded to thereby generate a modified set of updates. The federated ML computer model is updated based on the modified set of updates.

Federated learning with dataset sketch commitment based malicious participant identification

Aspects of the invention include computer systems, computer-implemented methods, and computer program products configured to integrate documentation knowledge with log mining data. A non-limiting example computer-implemented method includes determining a message-message relationship based on log message documentation and building a first subgraph based on the message-message relationship. The method further includes receiving a first message log entry having a message identifier and message field data. A second message log entry is correlated with the first message log entry based on at least one of the message identifier and the message field data. A second subgraph is built that includes the first message log entry and the second message log entry. The method includes building a graph that includes the first subgraph and the second subgraph.

Integrating documentation knowledge with log mining for system diagnosis

An approach for blockchain transaction processing using generative cryptograms for bi-lateral and Multi-lateral transactions may be provided. The approach may include, the use of decentralization ledger storage, negating the need of repetitive storage while preserving the need for data redundancy. A generative cryptogram may provide verification of processed blocks and file handles or storage address schemes of the ledger storage. The approach creates a cryptogram which is added to transaction processing, post processing and commits to the ledger. In the approach, the cryptogram is the only version of storage pointer and is utilized in subsequent transaction processing without the need for extensive compute and latest cryptogram is all that would be needed for transaction processing.

Generative cryptogram for blockchain data management

Embodiments of the invention are directed to annotating a log based on processing log documentation. Aspects include obtaining the data set and obtaining a set of templates, wherein one or more template of the set of templates includes a variable labeled as the data type. Aspects also include annotating the data set based on the set of templates, wherein the annotating includes labeling each value of the variable in the data set with the data type. Aspects further include creating an updated data set by replacing each value of the variable in the data set that is labeled with the data type with a new data value.

Removing data having a data type from a data set

A client computer may split a process into sub-processes, send each sub-processes to a different group of peers in a blockchain network, wherein each group has at least one peer from each essential organization in the blockchain network, receive processed sub-transactions from the peers in the blockchain network, validate each sub-transaction, and validate the transaction based on the validation of all sub-transactions, wherein all sub-transaction must be valid for the transaction to be valid.

Parallel processing of blockchain procedures

Aspects of the invention include computer systems, computer-implemented methods, and computer program products configured to perform message correlation extraction for mainframe operation. A non-limiting example computer-implemented method includes receiving a first message log entry having a message identifier and message field data. The first message log entry is pre-processed to determine the message identifier and to tokenize the message field data. A second message log entry is identified based on at least one of the message identifier and the tokenized message field data. The method further includes determining that the second message log entry is correlated with the first message log entry and providing an output comprising the message correlation between the first message log entry and the second message log entry.

Message correlation extraction for mainframe operation

A processor may record one or more transaction proposals regardless of validity of the one or more transaction proposals. The processor may separate a transaction validation into a series of steps. The processor may validate, asynchronously, one or more transactions respectively associated with each of the one or more transaction proposals based on the transaction validation.

Topological ordering of blockchain associated proposals

Embodiments of the invention are directed to annotating a log based on processing log documentation. Aspects include obtaining the log having a plurality of entries. Aspects also include creating a set of log entry templates by processing the log documentation associated with the log, wherein each log entry template includes one or more constants and one or more variables. Aspects further include annotating each of the plurality of entries based on the set of templates, wherein the annotating includes labeling each value of the one or more variables with a variable name.

Annotating a log based on log documentation

According to an aspect, a method includes searching for a correlated log identifier in a correlation database based on detecting a metrics-based anomaly. The method also includes providing, in a problem diagnosis, related log information associated with the correlated log identifier based on locating one or more log entries including the correlated log identifier in a same time window as the metrics-based anomaly. The method further includes searching for a correlated metric in the correlation database based on detecting a log-based anomaly and providing, in the problem diagnosis, related metric information associated with the correlated metric based on locating one or more metrics records including the correlated metric in the same time window as the log-based anomaly.

Correlation-based multi-source problem diagnosis

Techniques for refinement of data pipelines are provided. An original file of serialized objects is received, and an original pipeline comprising a plurality of transformations is identified based on the original file. A first computing cost is determined for a first transformation of the plurality of transformations. The first transformation is modified using a predefined optimization, and a second cost of the modified first transformation is determined. Upon determining that the second cost is lower than the first cost, the first transformation is replaced, in the original pipeline, with the optimized first transformation.

Streamlining data processing optimizations for machine learning workloads

Growing community of inventors

Sleepy Hollow, NY, United States of America

Lei Yu