International Business Machines Corporation

Shaanxi University of Technology

Tong Liu

Zhong Fang Yuan

Li Juan Gao

Xiang Yu Yang

Chen Gao

Yi Chen Zhong

Kun Yan Yin

He Li

De Shuo Kong

Li Ni Zhang

Yi Ming Wang

Si Tong Zhao

Ming Jin Chen

Yong Fang Liang

Yu Pan

Na Liu

Hai B Zou

Bin Shang

Yuan Yuan Li

Yuan Lin Yang

Chen Yu Chang

Si Heng Sun

Sarbajit Kumar Rakshit

Xin Peng Liu

Ke Zhang

Xiao Lin Sun

Lu Yu

Wen Jie Wang

Dan Zhang

Yao Liang Chen

Xue Ying Zhang

Yuan Yuan Zhang

Ling Chen

Dan Wu

Yuan Yuan Ding

Tian Ji Yang

Hong Bing Zhang

Zhuo Jp Cai

Hai Ji

Deng Xin Luo

Ci-Wei Lan

Xu Min

Jing Du

Guan Chao Li

Guang Qing Zhong

Elaine M Branagh

Teng Sun

Ya Dong Li

Pitipong Jun Sen Lin

Zheng Jie

Yun He Gao

Qiong Wu

Xueliang Zhao

Ya Juan Dang

Ziqiumin Wang

Peng HuangFu

Qiang He

Peng Yun Sun

Rong Fu He

Li Jia Lu

Wen Jie Hao

Ming Xia Shi

Yuan Li

Yu Zui Wy You

Frank Feng

Hui Dong

Yan Fen Liu

Ting Yao Liu

Teng Jiao Li

Hong Cui

Han Qiao Yu

Lin Feng

Yuan Zhang

Yazhou Wang

Jian Long Li

Provided are techniques for identifying and correcting model focus drift during model training. A model is trained using an original dataset with data, and the is classified into clusters. A representation is assigned to each of the clusters. A first visualization dashboard with visualizations is generated, where each visualization represents a first data distribution of an associated cluster using the representation assigned to that cluster. The model is fine-tuned using a fine-tune dataset. A second visualization dashboard is generated by updating each visualization, where each visualization represents a second data distribution of the associated cluster. It is determined that a cluster of the clusters has focus drift based on changes between the first data distribution and the second data distribution. The focus drift is corrected by: adding the data of the cluster to the fine-tune dataset to form a combined dataset and fine-tuning the model using the combined dataset.

Identifying and correcting model focus drift during model training

Provided are a computer program product, system, and method for training and using a vector encoder to determine vectors for sub-images of text in an image to subject to optical character recognition. A vector encoder is trained to encode images representing text into vectors in a vector space. Vectors of images representing similar text have a high degree of cohesion in the vector space. Vectors of images representing dissimilar text have a low degree of cohesion in the vector space. An input image is processed to determine sub-images of the input image that bound text represented in the input image. The sub-images are inputted to the vector encoder to output sub-image vectors. The vector encoder generates a search vector for search text. Optical character recognition is applied to at least one region of the input image including the sub-images having sub-image vectors matching the search vector.

Training and using a vector encoder to determine vectors for sub-images of text in an image subject to optical character recognition

The embodiments of the present disclosure disclose a computer-implemented method, computer system and a computer program product for detecting and predicting an abnormal log event. In the method, a current event cluster from a plurality of event clusters for a log line in a log file is determined. The plurality of event clusters include at least one abnormal event cluster. Then, a time of event transition from the current event cluster to at least one abnormal event cluster is predicted.

Abnormal log event detection and prediction

According to one embodiment, a method, computer system, and computer program product for building an acoustic model is provided. The present invention may include performing contrastive pre-training of the acoustic model; building a dataset classifier using prompt engineering; performing a prediction process; and performing zero-shot audio prediction using the pre-trained acoustic model.

Adaptable acoustic model built with limited labeling data

Embodiments of the present invention provide an approach for compressing data, and more particularly, to large-scale text data encoding and compression using absolute overfitting on pre-trained language models. Large-scale data is parsed into sentences. A unique token is generated for each sentence to form a token list. A generative (or compression) model is trained from the tokens in the token list to produce the corresponding sentence of each token through absolute overfitting of a pre-trained language model. The compressed text data is stored as the token list and generative model, resulting in a storage space savings.

Large-scale text data encoding and compression

A method for conversational query resolution includes receiving a query input from a user. The query input is decomposed into a plurality of tasks. A knowledge graph is queried to identify one or more relevant entities based on at least one of the plurality of tasks. A vector database is searched to identify one or more text chunks that correspond to the one or more relevant entities. Content relevant to at least one of the plurality of tasks is identified from the one or more text chunks. An answer is generated based on the identified content.

Graph and vector usage for automated QA system

Information extraction and image restructuring includes generating semantic vectors to encode portions of text extracted from a document. For each semantic vector a semantic similarity between a schema key and other text encoded therein is determined based on their respective positions within the document. An enhanced NLP model is created using the semantic vectors, each labeled according to the semantic similarity. The text, including schema key, are re-encoded as a key and candidate vectors. Key-value pairs are generated by matching the key vector with a predetermined number of candidate vectors. The enhanced NLP model, using prompt learning, is repurposed to perform a next-sentence prediction that predicts which of the candidate vectors is logically related to the schema key. Based on the next-sentence prediction, the discrete portion of text identified as the schema key and portion of text determined to be logically related thereto are output.

Information extraction and image re-ordering using prompt learning and machine-reading comprehension

An example operation may include one or more of generating a plurality of bounding boxes at a plurality of content areas in an image corresponding to a plurality of pieces of text within the image, converting the plurality of bounding boxes into a plurality of bounding box vectors based on attributes of the plurality of bounding boxes, training a machine learning model to transform a bounding box into a location in vector space based on the plurality of bounding box vectors, and storing the trained machine learning model in memory.

Visualization of the impact of training data using bounding box vectors to train a machine learning model

An example operation may include one or more of executing a machine learning model on training data, where the training data comprises a plurality of word strings, identifying words within the training data that are extracted by the machine learning model during the executing, determining a color for the machine learning model based on the identified words and a predefined mapping of words to colors, and rendering, via a user interface, a label associated with the machine learning model in the determined color for the machine learning model.

Visualization of the impact of training data

In order to perform a semantic search based on a graph database, sets of nodes are selected from a plurality of nodes in a graph database. A set of nodes semantically matches a keyword in a natural language query. At least one target node is identified in the sets of nodes. A path is selected from candidate paths based on similarities between the candidate paths and a plurality of paths in the graph database. A graph query for retrieving information from the graph database is generated based on the selected path and the query target.

Semantic search based on a graph database

A given segment of computer code is obtained and modified to produce one or more inefficient versions of the given segment of computer code in comparison to the given segment of computer code. A code parse tree is generated for the given segment of computer code and each inefficient version of the given segment of computer code. Model embeddings are generated based on the generated code parse trees and a diffusion model is trained based on the generated model embeddings.

Code performance optimization based on an information-enhanced diffusion model

An approach is provided for optimizing a feedback-type question answering process. A training set is constructed to detect missing information of a question. A natural language generation model is trained using the missing information. The natural language generation model is executed to generate a rhetorical question. A response to the rhetorical question is combined with the question to generate an input to a language processor. A new question is generated. The new question is applied to a document library. A final answer is generated.

Question answering information completion using machine reading comprehension-based process

A method, system, and computer program product for reconstructing training data and building a new incremental learning model with the reconstructed training data that can be further trained. The method may include receiving new data to be inputted into a previously trained machine learning model, where the previously trained machine learning model has inaccessible training data. The method may also include generating simulated training data using a reverse form of the previously trained machine learning model. The method may also include verifying the simulated training data. The method may also include creating a new machine learning model using the simulated training data, where the new machine learning model includes a same structure as the previously trained machine learning model. The method may also include inputting the new data into the new machine learning model, where the new machine learning model is further trained with the new data.

Incremental learning through reverse data distribution

Analysis of edge closures of metal surface particles based on a graph structure.

Analyzing edge closures of metal surface particles based on graph structures

A computer-implemented method, according to one embodiment, includes: receiving an image having characters that correspond to a language, and using a text recognition algorithm to determine a first language believed to correspond to the characters. A first confidence level associated with the first language is also computed, and a determination is made as to whether the first confidence level associated with the first language is outside a predetermined range. In response to determining that the first confidence level associated with the first language is not outside the predetermined range, the first language is output as the given language. The text recognition algorithm is trained using a simple shallow neural network and a generated mixed language corpus. The generated mixed language corpus is formed by: randomly sampling libraries having vocabulary and/or characters therein, and combining the randomly sampled vocabulary and/or characters to form the generated mixed language corpus.

Rapid language detection for characters in images of documents

Growing community of inventors

Xi'an, China

Tong Liu