Electronics and Telecommunications Research Institute

Ran Han

Hyun Woo Kim

Jeon Gue Park

Hwa Jeon Song

Eui Sok Chung

Euisok Chung

Yoo Rhee Oh

Byung Hyun Yoo

Hwajeon Song

Byunghyun Yoo

A domain adaptation procedure, such as fine-tuning training, is required to utilize a large-capacity PLM for a specific domain. Attempts in existing research have been made to improve performance of a PLM through domain adaptor technology based on an N-gram in order to reduce errors on the basis of the results of domain text error analysis of the PLM. Proposed is a method of selecting a semantic chunk through a domain semantic chunk graph and PageRank based on the existing domain adaptor research, with an N-gram as the semantic chunk. Proposed is also a method of domain-adapting a large-capacity PLM using semantic chunk dynamic weight masking, which reflects an output value of a PLM rather than simply integrating embedding values of semantic chunks, in a semantic chunk domain adaptor technology.

Method of domain-adapting large-capacity pre-trained language model using semantic chunk dynamic weight masking

A knowledge increasing method includes calculating uncertainty of knowledge obtained from a neural network using an explicit memory, determining the insufficiency of the knowledge on the basis of the calculated uncertainty, obtaining additional data (learning data) for increasing insufficient knowledge, and training the neural network by using the additional data to autonomously increase knowledge.

Method and apparatus of increasing knowledge based on uncertainty in neural networks

Growing community of inventors

Daejeon, South Korea

Ran Han